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INTRODUCTION 


A book for ‘project researchers’ 


Social research is no longer restricted to a small elite of professionals and full- 
time researchers. It has become the concern of a far greater number of people 
who are faced with the prospect of undertaking small-scale research projects as 
part of an academic course or as part of their professional development. The 
aim of this book is to present these ‘project researchers’ with practical guidance 
and a vision of the key issues involved in social research. It attempts to provide 
them with vital information that is easily accessible and which gets to the 
heart of the matter quickly and concisely. In doing this, the book is based on 
three premises: 


1 


2 


Most of what needs to be known and done in relation to the production 
of competent social research can be stated in straightforward language. 

The foundations of good social research depend on paying attention to 
certain elementary factors. If such factors are ignored or overlooked, the 
research will be open to criticism and serious questions may be raised about 
the quality of the findings. Good research depends on addressing these key 
points. The answers may vary from topic to topic, researcher to researcher. 
There may be no one ‘right’ answer, but the biggest possible guarantee of 
poor research is to ignore the issues. 

Project researchers can safeguard against making elementary errors in the 
design and execution of their research by using checklists to assure them- 
selves that they have attended to the ‘minimum’ requirements and have 
not overlooked crucial factors associated with the production of good 
research. 
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There is no single pathway to good research: there are always options and 
alternatives. At any stage of any enquiry, researchers need to take decisions, 
make judgements and use discretion in order to successfully complete a proj- 
ect. One of the biggest decisions to be made concerns the choice of research 
strategy, and the first section of this book outlines a variety of strategies that 
can be used. These are not the full range of possibilities that exist because 
this would be beyond the scope of this book and would include strategies 
more appropriate for large-scale, long-term and very costly investigations. The 
focus, instead, is on the range of strategies that are well suited to the needs 
of project researchers engaged with small-scale, low-budget research being 
conducted over relatively short periods of time. 


What are research strategies? 


A strategy is a plan of action designed to achieve a specific goal. The concept 
has military origins relating to the role of generals: the way they have a broad 
overview of operations, the way they plan specific combat tactics as part of a 
bigger plan for overall victory, and the way they rely on careful planning to 
achieve their goal. Applied to social research, the notion of a strategy retains 
the same essential components. It requires: 


e an overview of the whole project that uses ‘the bigger picture’ as the basis for 
deciding how to approach the research (a research paradigm); 

e a carefully constructed plan of action that is rationally designed and likely 
to offer the best prospects of success (a research design); 
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e aspecific goal that can be achieved and which is clearly identified (a research 
problem). 


A research strategy is different from a research method. Research methods 
are the tools for data collection - things like questionnaires, interviews, 
observation and documents (which are outlined in Part II of this book). 
Methods are the equivalent of a microscope when used by a scientist, a therm- 
ometer when used by a medic, or a telescope when used by an astronomer. 
They do a job. They are types of equipment that allow relevant data to be 
collected. 

In practice, certain methods tend to be associated with certain strategies. For 
example, the use of a survey strategy is often linked with the use of question- 
naires as the method for data collection. This is because the strategy and the 
method tend to work well together. However, the choice of research strategy 
does not actually dictate the choice of any specific research method and, in 
principle, social researchers can use a range of methods with any given strategy. 
There is always an element of choice. 


Good practice: the distinction between strategies and methods 

The project researcher needs to be clear about the difference between strat- 
egies and methods, and should be clear about the grounds for their choice of 
strategy. 


Which strategy to choose? 


There is no single strategy that can be recommended as the ‘best’ in all circum- 
stances. The choice of strategy, instead, depends on identifying one that works 
best for the particular research project in mind. It is a matter of ‘horses for 
courses’ — choosing a strategy that is ‘fit for purpose’ in relation to the particular 
thing the research is trying to achieve. 

To decide which strategy is likely to work best, the researcher needs to 
consider three key questions: 


e Is it suitable? 
e Is it feasible? 
e Is it ethical? 


The answers to these questions might point to a particular strategy as the 
favourite one, clearly standing out as the most suitable for the task at hand. 
However, it is also possible that no single strategy stands out as the winner on 
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all points. In this case, the researcher is faced with making a choice between 
two or more alternatives, each of which comes with its respective strengths 
and weaknesses in relation to the specific nature of the research project being 
considered. Each choice brings with it a set of advantages and disadvantages. 
Gains in one direction will bring with them losses in another, and the social 
researcher has to live with this. Whichever decision is made, however, it is 
important that the choice of strategy can be justified in terms of being feasible, 
being ethical, and as providing suitable kinds of data for answering the 
research question. 


Good practice: the choice of strategy is a matter of ‘horses for courses’ 


The researcher should choose a strategy that is likely to be successful in 
achieving the aims of the research, and be able to justify the choice of this 
strategy clearly and explicitly. 


Is it suitable? 


In themselves, research strategies are neither ‘good’ nor ‘bad’, nor are they 
‘right’ or ‘wrong’. It is only in relation to how they are used that they take on 
the mantle of being good or bad, right or wrong. They can only be judged in 
relation to the purpose for which they are used and, reflecting this point, it is 
better to think of strategies in terms of how useful they are and how appropriate 
they are. Thought of in this pragmatic way, the justification for the choice of 
any specific strategy must depend on having a clear vision of the particular 
purpose for which it is being used. Unless we know what the research is trying 
to achieve, it is impossible to judge whether it is likely to be useful or appropri- 
ate. Table 0.1 indicates how the research strategies outlined in Part I can be 
valuable for different kinds of research purpose, and this should provide some 
initial guidance on the question of whether a particular strategy is suitable for 
the kind of research that is envisaged. 


Table 0.1 Research strategies and research purpose; some links 


Strategy Purpose of research 


Surveys measure some aspect of a social phenomenon or trend 


gather facts in order to test a theory 


Case studies understand the complex relationship between factors as they 


operate within a particular social setting 


Experiments e identify the cause of something 
observe the influence of specific factors 


Ethnography describe cultural practices and traditions 
interpret social interaction within a culture 


(Continued overleaf) 
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Table 0.1 Continued 


Strategy Purpose of research 


Phenomenology describe the essence of specific types of personal experience 


e understand things through the eyes of someone else 


Grounded theory e clarify concepts or produce new theories 
e explore a new topic and provide new insights 


Action research e solve a practical problem 
e produce guidelines for best practice 


Mixed methods e evaluate a new policy and gauge its impact 
compare alternative perspectives on a phenomenon 
combine aspects of the other strategies 


Is it feasible? 


The choice of strategy needs to take into account certain practical aspects of 
conducting research. It is vital, for instance, that the researcher can get access 
to the kind of data sources that the strategy requires. The researcher must be 
able to gain access to the kinds of people, contexts, events and documents that 
will be crucial for the success of the research. This might be a matter of getting 
the necessary authorization to contact people, enter locations or view docu- 
ments. It might be a matter of getting agreement from people or organizations 
to collaborate with the research. Or, it might be a matter of having the 
resources for specialist equipment or travel to fieldwork locations. Whatever 
form it takes, the point is that access to data sources is crucial for the choice of 
research strategy. 

Another practical consideration is the need to meet deadlines and to com- 
plete research within specified time constraints. Project researchers generally 
face deadlines for the completion of the research and it would be wrong to 
choose a strategy which could not possibly allow such deadlines to be met. 
Some strategies are relatively predictable in terms of their time-span and lend 
themselves to fairly tight time-planning (e.g. surveys, case studies, experi- 
ments). Others are less predictable (e.g. grounded theory) or tend to take a 
relatively long time (e.g. ethnography). This must be taken into account in 
terms of the feasibility of using specific strategies. 

When choosing the strategy researchers should also bear in mind the audi- 
ence for the research and the particular research community within which the 
researcher is working. Different subject areas and disciplines tend to have dif- 
ferent perspectives on research and, in practice, this means that they generally 
have a preference for certain styles of research rather than others. Project 
researchers need to be aware of this. In particular, those who need to conduct a 
small-scale piece of research as part of an academic qualification might be 
advised to make their choice of strategy mindful of the traditions of their own 
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specific discipline and the predispositions of the people who are going to 
evaluate their research — their tutors and examiners. 


Is it ethical? 


Research ethics are an over-riding concern when it comes to the choice of strat- 
egy. It is hard to overstate the importance of research ethics in the context of 
contemporary social research and, for this reason practical guidance on 
research ethics is provided in Appendix 1 on p. 331. These guidelines are based 
on some core principles of research ethics, the first of which is that no-one 
should suffer harm as a result of participation in the research. The notion of 
‘harm’ in this context is wide-ranging and it calls on the researcher to think 
carefully at the outset of the research about the possible ways in which partici- 
pants might suffer as a consequence of their involvement with the research. 
The emphasis is very much on the possibility of harm, rather than the actual- 
ity, so the researcher needs to consider the likelihood of things occurring 
which could cause harm if they were to happen. 

There are certain standard measures which researchers are expected to put 
in place to minimize the risk of harm. For social researchers these normally 
involve guarantees that: 


e participants will remain anonymous; 

e data will be treated as confidential; 

e participants understand the nature of the research and their involvement; 
e participants voluntarily consent to being involved. 


There is an expectation, as well, that researchers act professionally and with 
integrity. This expectation can be found in the codes of research ethics that 
exist for all researchers within their specific disciplines or professions, and 
which are freely available online. These codes share the view that: 


e researchers must operate within the law; 

e in their dealings with participants they are expected to be open and honest, 
and not to engage in deception; 

e in their dealings with colleagues and the research community they are 
expected to uphold good standards of science and not ‘fiddle the results’; 

e in the collection of data they are expected to take matters of personal safety 
seriously and not expose themselves or co-workers to unacceptable danger 
in the pursuit of data. 


There are exceptions to some of these ethical ‘rules’ which might be acceptable 
under certain circumstances but, from the project researcher’s point of view, 
it is probably wise to start by considering them as baselines which need to be 
adhered to. The choice of strategy should allow the research to abide by these 
ethical ‘rules’. A strategy should not be chosen if it likely to cause harm to 
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those who participate in the research or if it compromises the integrity of the 
researcher. No matter how beneficial a strategy might appear to be in terms of 
the quality of the information it could supply, a concern with research ethics 
is more important. 


Link up with Research ethics: a practical guide, Appendix 1 ` 
"ee 
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Checklist for the choice of researc 


When choosing a research strategy you should feel confident about answer- 
ing ‘yes’ to the following questions: v 


Factors to be 
considered 


Suitability Has the purpose of the research been clearly identified? 

Will it produce Is there a clear link between the purpose of the research 

appropriate kinds and the chosen strategy? 

of data? Will the strategy produce findings that can answer the 
research question(s)? 


Feasibility Is there sufficient time for the design of the research, 
Can it be done? collection of data and analysis of results? 
Are sufficient resources available to cover the costs of 
the research (e.g. travel, printing)? 
Is it possible and practical to gain access to necessary 
data (people, events, documents)? 
Will the chosen strategy be favoured by the key 
evaluators of the research? 


Ethics Can | avoid any harm to participants resulting from their 
Will it allow meto involvement in the research? 
be ethical in my Can | get informed consent from potential participants? 
dealings with Will the strategy permit me to work within an appropriate 
participants? code of research ethics? 
Can | guarantee the confidentiality of the information 
given to me during the research? 


© M. Denscombe, The Good Research Guide. Open University Press. 


SURVEYS AND 
SAMPLING 


What is a survey? e When are surveys useful? e Types of survey 
e Response rates e Sampling e Probability sampling techniques 
e Non-probability sampling techniques e Which sampling technique 


should be used? e Size of the sample e Advantages of surveys and 
sampling e Disadvantages of surveys and sampling e Checklist 
for surveys and sampling 


What is a survey? 


When something is surveyed, it is ‘viewed comprehensively and in detail’, 
and the purpose of doing a survey is generally to ‘obtain data for mapping’. 
This is clear in the case of geographical surveys and ordnance surveys which 
map out the landscape or the built environment of roads and buildings. The 
principles, though, have been used to good effect on mapping out the social 
world as well as the physical world and, indeed, surveys have emerged in 
recent times as one of the most popular and commonplace approaches to 
social research. Such social surveys share with their physical counterparts three 
crucial characteristics: 


1 Wide and inclusive coverage. Implicit in the notion of ‘survey’ is the idea that 
the research should have a wide coverage — a breadth of view. A survey, in 
principle, should take a panoramic view and ‘take it all in’. 
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2 Ata specific point in time. Surveys provide a snapshot of how things are at a 
specific point in time. In most cases this is useful for ‘bringing things up to 
date’ and providing information about the current state of affairs. However, 
there are also occasions when researchers do an historical survey to show 
how things used to be at a particular point in the past, or even use a sequence 
of surveys as the basis for tracking changes over a period of time. 

3 Empirical research. Because ‘to survey’ carries with it the meaning ‘to look’, 
survey work inevitably brings with it the idea of empirical research. It 
involves the idea of getting out of the chair, going out of the office and 
purposefully seeking the necessary information ‘out there’. 


These three characteristics of the survey approach involve no mention of spe- 
cific research methods. It is important to recognize this point. The survey 
approach is a research strategy, not a method. Although there are certain 
methods which are popularly associated with the use of surveys there are actu- 
ally a variety of methods including questionnaires, interviews, documents 
and observation that can be used as part of a survey approach. 


When are surveys useful? 


Surveys are used to best effect when the researcher wants factual information 
relating to groups of people: what they do, what they think, who they are. This 
is not the only use of surveys but it is the most common use among social 
researchers and it reflects the particular strengths of the survey approach. The 
first of these is that surveys are good at getting information about a large number 
of people: they are well suited to the collection of mass data. Second, the survey 
approach works best with clear and narrow targets in terms of information it is 
trying to gather. The approach lends itself to dealing with specific issues and is at 
its best when the researcher knows in advance precisely which factors are 
important and what kind of information is needed. Third, surveys are best 
suited to gathering data on relatively straightforward, relatively uncomplicated 
facts, thoughts, feelings or behaviours. The breadth of coverage of surveys 
means that it is hard to incorporate the depth of investigation needed for really 
complex issues. And fourth, surveys are particularly good when looking for 
patterns of activity within groups or categories of people (rather than indi- 
viduals). They are very useful for linking findings with specific social classes, 
age-groups, sexes, ethnic backgrounds, etc. These four aspects of the survey 
approach are evident in the familiar use of surveys in relation to things like: 


e opinion polls and voting behaviour; 
e consumer preferences and customer satisfaction; 
e health-related behaviour, diet, smoking and alcohol consumption; 
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e evaluation of educational courses and new innovations; 
e TV viewing and audience ratings. 


The survey approach is less useful when it comes to research on sensitive and 
complicated matters. Surveys do not lend themselves so well to studying issues 
in depth or teasing out the intricacies and subtleties of certain facets of social 
life. Indeed, other research strategies can prove more valuable in terms of 
dealing with: 


e complex issues or sensitive subjects (interviews, ethnography); 
e depth insights or holistic overviews (case studies); 

e practical problems (action research, experiments); 

e personal experiences (phenomenology); 

e developing theories (grounded theory). 


Types of survey 


The major development of the twenty-first century, so far, is recognition 
that the use of all survey modes continues, and none seem likely to disap- 
pear. Each has strengths and weaknesses, allowing it to do things other 
modes cannot do. 

(Dillman 2007: 495) 


Postal surveys 


Probably the best-known kind of survey is that which involves sending ‘self- 
completion’ questionnaires through the post. This generally involves a large- 
scale mailing covering a wide geographical area. 

Three things need to be borne in mind when considering the use of a postal 
survey. First, there is the cost. The printing, packing and postage costs can 
quickly mount up, particularly with larger surveys. Project researchers with 
limited resources need to think carefully about this. Second, there is the time 
delay. Planning and preparation take time, as they do with all research, but 
there are specific delays that stem from the turnaround time involved with 
posting and returning questionnaires as part of a postal survey. In practice, 
the data collection is likely to take a number of weeks to complete even after 
the questionnaires have been posted out. Third, there is the response rate. The 
proportion of people who respond to postal surveys is generally quite low. 


Internet surveys 


Internet surveys can take the form of: 
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e an email questionnaire where the questions are sent as part of the email itself; 

e a questionnaire sent as an attachment to an email; 

e a web-based questionnaire where the questionnaire is a web page located on 
an Internet site waiting for people who visit the site to complete it. 


Internet surveys provide a fast and cheap alternative to postal surveys and, 
for this reason, they can prove to be an attractive proposition for project 
researchers who need results quickly and on a restricted budget. There is no 
travel, no venue needed and no specialist equipment required. The use of 
emails for distributing the survey can be virtually free and, although web- 
based questionnaires could cost quite a lot in terms of fees for designing and 
hosting the questionnaire, it is quite easy to find low-cost Internet survey sites 
that provide an effective service and whose charges should fall well within 
the resources available to most project researchers. The data supplied via web- 
based questionnaires, as a bonus, arrive in a format that is ready for analysis. 
Data from Internet questionnaires can be downloaded automatically into 
spreadsheets, databases or statistics packages. There are potentially no data 
entry costs (and no transcription errors). 

Internet surveys also reduce the turnaround time between sending out a 
questionnaire and receiving a completed response. The researcher gets the 
replies more quickly because delivery of the questionnaire can become virtually 
instantaneous and also because people get round to answering the questions 
sooner than with conventional surveys. 

Such savings in time and money, furthermore, do not mean that the 
researcher has to make sacrifices in terms of the quality of data. Recent evidence 
would suggest that the quality of data obtained through Internet survey 
research is not significantly different from that obtained using more trad- 
itional methods. As the Internet increasingly becomes part of everyday life 
for most people in developed societies, there is little evidence that respon- 
dents provide different kinds of answer or supply different amounts of infor- 
mation as a consequence of the online data collection method (McCabe 2004; 
Denscombe 2006, 2008, 2009; Lozar Manfreda et al. 2008). 


Telephone surveys 


Telephone surveys, like Internet surveys, are now in widespread use in social 
research. They provide an alternative to postal and Internet surveys as a means 
of collecting questionnaire data, and they can also be used for conducting 
interviews. They can be relatively labour-intensive compared with postal and 
Internet surveys but the costs of the calls themselves need not be great. Local 
and national calls on landlines are not generally expensive and even inter- 
national calls can now be made via the Internet virtually free of cost. Calls to 
mobile phones, however, can create hefty bills. There are no significant delays 
with the return of data using a telephone survey because the responses are 
instantaneous, and this is a significant advantage over postal and Internet 
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surveys. Weighed against this, phone interviews are unlikely to last beyond 
20-30 minutes and there can be problems ensuring that contact is made with 
the right person; it is not always easy to verify the identity of the person at the 
other end of the line. 

In the past it was argued that telephone surveys would produce a biased 
sample because poorer people were less likely to own a phone. This criticism 
does not really apply today. Most people aged over 18 years can be contacted 
directly by telephone, which means that there is the prospect of contacting a 
representative sample when conducting surveys by phone. It was also argued 
in the past that people might be less inclined to tell the truth when answering 
questions over the phone. However, recent evidence shows that people today 
are as honest and open on the phone as they are when answering question- 
naires or being interviewed face-to-face (Dillman 2007). 

Telephone surveys have a particular advantage when used for interviewing. 
Researchers do not have to travel to all parts of the country to conduct the 
interviews — they only have to pick up a phone. This means that choice of 
potential interviewees no longer has to take into consideration things like 
the distance between locations and the researcher costs incurred through 
travel. Phone surveys can also target particular geographical areas, something 
which postal surveys can do but which Internet surveys cannot. Researchers 
can select the area they wish to survey and identify the relevant dialling code 
for that area. They can then contact phone numbers at random within that 
area using random-digit dialling, where the final digits of the phone numbers 
are produced by computer technology which automatically generates num- 
bers at random. 


/\ Caution 


Cell phones pose something of a problem for telephone surveys. 
Increasingly, people are using ‘mobiles’ instead of — not just in 
addition to — conventional land-line phones and it is likely that, as 
things progress, more people will come to rely exclusively on their cell 
phone and cease to use a household land-line number. As a con- 
sequence, telephone surveys will face a new challenge in terms of 
reaching a representative sample of the population. This is because 
cell phone numbers are not allocated by geographical location (in the 
UK), which means that the researcher cannot use prior knowledge 
about the social background or geographical location of any cell phone 
user as the basis for including them in a survey. 
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Group-administered surveys 


Group-administered surveys are conducted in person. They rely on the 
researcher undertaking the survey by being on site to distribute and collect the 
research instrument — normally a questionnaire. The groups generally already 
exist. Rather than get a group of people together specifically for the purpose, it 
is easier from the researcher’s point of view to use naturally occurring groups. 
For example, a survey of young people could involve the distribution of ques- 
tionnaires in schools to whole classes — possibly 30 or more people at a time. 
The same approach could be taken with work-groups, community meetings or 
any occasions where there will be a (naturally occurring) collection of people 
who are suitable for the purposes of the research. 

For practical purposes the group-administered survey tends to involve fairly 
small numbers. It does not really lend itself to use with thousands of people 
in a sports arena. It is, though, particularly well suited to small-scale research. 
It is relatively easy to organize and can often draw on the researcher’s personal 
knowledge about the context of the research. In terms of turnaround time, 
again, this form of survey has advantages because the questionnaires are 
generally distributed, completed and returned in one process. The personal 
contact involved in the process means that response rates tend to be consider- 
ably higher than with postal, Internet or phone surveys. However, because it 
requires the researcher to be physically present, it forfeits any savings in time 
and money that might be gained by using the post, the Internet or the 
telephone to distribute the survey. It can be labour-intensive and, depending 
on the locations for research, can incur significant travel costs. 


Link up with Cluster sampling, p. 29 ; 
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Face-to-face surveys 


Face-to-face surveys involve direct contact between the researcher and indi- 
vidual respondents and they normally make use of various forms of question- 
naires or interviews as their data collection method. Contact with respondents 
can arise through approaches made by the researcher ‘in the street’. The sight 
of the market researcher with his/her clipboard and smile is familiar in town 
centres. Or the contact can be made by calling at people’s homes. Sometimes 
this will be ‘on spec’ to see if the householder is willing and able to spare the 
time to help with the research. On other occasions, contact will be arranged 
in advance by letter or phone. 

The face-to-face survey is a more expensive way of collecting data than the 
use of the post, Internet or telephone. Researcher time and travel costs are 
considerable. Weighed against this there is the prospect of probing beneath 
the surface to get a bit more depth to the information that is obtained. Plus, 
this kind of survey with its face-to-face contact offers some immediate means 
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of validating the data. The researcher can sense if he/she is being given false 
information in the face-to-face context in a way that is not possible with 
self-completion questionnaires or telephone surveys. And, with this kind of 
survey, response rates pose less of a problem. Partly this is because the face-to- 
face contact allows the researcher to use his/her interpersonal skills and powers 
of persuasion to encourage the potential respondent to take part in the survey. 
Partly it is because with face-to-face surveys the researcher can continue 
making contacts until he/she has accumulated the total number of responses 
that are required. 


Link up with Quota sampling, p. 34 A 
Observational surveys 


Observational surveys are used to make detailed, factual records of specific 
behaviours, events and settings, and they generally employ a systematic 
‘observation schedule’ to gather the data. In this respect they are quite different 
from the other forms of survey outlined so far. Instead of relying on self-reports 
based on what people say they do and say they think, observational surveys 
focus on what people actually do. What matters is not some stated preference 
or some stated intention to do something but what behaviour actually takes 
place that can be observed and systematically recorded. 

Observational surveys can also be used to look at particular events - when 
they occur, how often they happen, how long they last. For example, an 
observational survey could provide precise factual data about queuing at super- 
market check-outs; how long shoppers spend standing in line, the average 
number of people in the queue, and the peak times in the week for queuing. 
When it comes to any decision-making linked to such events the factual data 
obtained from the observational survey is likely to be far more persuasive than 
any subjective impressions. 

A third use of observational surveys is for collecting information about the 
social settings within which particular activities take place. They can provide 
a valuable means of recording features of the social context and physical 
environment — factors that can have a significant impact on the situation. 
Observation of school classrooms, for example, could include a record of the 
size of the room, the number of students, the ambient temperature, the time 
of the day, etc., each of which could have a bearing on the behaviour and the 
events that take place during a lesson. 

Observational surveys can serve the purposes of small-scale research very 
well. They can be used to investigate specific settings (such as hospital wards 
or classrooms), need not involve the use of sophisticated equipment and are 
normally conducted in person by the researcher. As with the other in-person 
types of survey, observational surveys can be rather labour-intensive, and they 
can also involve the costs of travel if the locations in which the observations 
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are to take place happen to be some distance from the researcher’s home 
base. Access to settings is an important consideration for this kind of survey. 
Depending on whether the setting is a public space or not, the researcher is 
likely to need prior authorization for access to the setting, might well need 
ethical approval before the research can commence, and will need to be very 
sensitive about what gets recorded. If, for example, the researcher wants to 
move beyond the use of pen and paper technology to record his/her observa- 
tions and chooses to use a camera or video recording instead, then there are 
issues of propriety relating to the collection and storage of images. In places 
like schools and hospitals this can be extremely important. 

Observational surveys are generally designed to record the kind of things 
that exist objectively and which can be easily recognized. They do not work 
well if the researcher needs to use interpretation or exercise discretion in order 
to know what to record. They are well suited to producing quantitative data 
and are particularly valuable in the context of research dealing with practical 
problems. These points can be seen in recent examples of observational studies 
that have focused on: 


e use of seat belts in cars; 
e styles of teaching in school classrooms; 


e shoppers’ responses to product displays in stores. 


Link up with Systematic observation, p. 199 


© 
Surveys of documents 


Although most social surveys focus on getting information from people, the 
survey strategy can be applied to documents as well. Historic archives, news- 
papers, company reports, and committee minutes are just some of the possible 
sources that can be surveyed. Photographs and films are instances of visual 
kinds of documents which also lend themselves to being researched using a 
survey approach. 

The main consideration with this kind of survey is access to the documents. 
At one level this involves the authorization to view the documents. The feasi- 
bility of this form of survey research clearly depends crucially on whether 
the researcher can get authorized access to the documents. Depending on 
the type of document and who owns it, researchers might be allowed free 
access, access might be restricted, or it might be completely denied. For many 
purposes researchers can use documents that are in the public domain. If 
the survey covers archives that are publicly available through, for instance, 
libraries, museums, Companies House or the Public Records Office, there 
should be no restrictions, and little or no cost associated with access. On the 
other hand some documents might only be available at a price — the charges 
reflecting either an administration fee or the commercial value of the informa- 
tion. Access to some documents, such as health records, might be restricted 
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on ethical grounds. And where documents are commercially, militarily or 
politically sensitive, they are likely to be kept secret with all access being 
denied. 

In another sense, the issue of access relates to the costs of travel involved 
with a survey of documents. Sometimes surveys of documents can be con- 
ducted with minimal cost. With an increasing volume of documentary data 
being available online and freely downloadable, there is the prospect of a very 
convenient form of survey with no travel costs incurred. If the documents are 
not downloadable, however, travel costs come into the equation. These travel 
costs might be considerable if the documents are in a national archive rather 
than a local museum. The researcher might need to travel from one part of the 
country to another to visit the location of the documents, and do so on a 
regular basis during the period of the investigation. 


Link up with Documents, Chapter 12 
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Response rates 


When conducting a survey there is a strong likelihood that some of those who 
are contacted with requests for information will not co-operate. The aim of a 
good survey is to keep such non-responses to a minimum and to achieve the 
highest response rate that is possible in relation to the kind of research being 
conducted. To do this, it is important for researchers to take into consideration 
five things that will have a bearing on the likely response rate achieved by the 
survey they have in mind: 


e Nature of respondents (age, sex, disability, literacy, employment status, etc.). 
Certain kinds of people are less inclined than others to spare the time and 
make the effort to comply with requests to help with surveys. Busy people 
can ill afford the time. Others with more time on their hands, those who are 
retired, for instance, might be more inclined to get involved. People with 
communication disadvantages, those with reading or hearing difficulties, 
are less likely than others to get involved unless special attention is devoted 
to their needs. 

e Subject of research (sex, race, religion, politics, income). Certain subjects are 
taboo and others are sensitive. If the investigation touches on intimate 
matters or embarrassing topics, there is every likelihood that the response 
rate will be low. 

e Researcher identity (age, sex, social class, ethnicity, clothes, accent). Where 
the survey involves face-to-face contact between the researcher and the 
respondent, the researcher needs to avoid, as far as is possible, presenting 
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himself or herself in a way that will be perceived as threatening or unwhole- 
some by the potential respondent. 


Link up with The interviewer effect, p. 178 ` 
"ee 

e Nature of contact. The way in which people are approached in connection 
with a survey has an influence on the response rate. Postal surveys in which 
the potential respondent receives the questionnaire ‘cold’ without any prior 
notice are prone to very low response rates. A 10 per cent response rate 
would not be surprising, and it could easily be lower. The use of prior notifi- 
cation and the use of follow-up reminders, however, can raise the response 
rate possibly to the 70 per cent region (Dillman 2007). And this is true for 
Internet and telephone surveys as well. So, the researcher needs to have a 
strategy for establishing contact and providing timely reminders as an inte- 
gral part of the survey approach. 

e Social climate (free speech). The right to free speech is obviously a factor that 
will influence people’s willingness to collaborate with research and to sup- 
ply honest and full answers. But this is not simply a matter of legal rights 
in a democratic society. There are situations in organizations and other 
social settings where potential respondents may not feel free to speak their 
thoughts. A threatening climate, wherever it exists and whatever its cause, 
can reduce the response rate. 


Non-response bias 


There are two types of non-response, both of which can lead to bias in the 
survey. First, there is non-response through refusal. If there are grounds for 
believing that those who refuse to co-operate are consistently of a different 
type from those who tend to provide responses (e.g. in terms of age, sex, 
gender, social class, religion) and if this difference is relevant to the matter at 
hand for the research, then there is the likelihood of a bias in the survey 
findings. The survey will systematically over-represent findings from the 
groups who are more likely to respond and, by the same token, systematic- 
ally under-represent findings from those groups who are less likely to 
respond. Second, there is non-response stemming from non-contact. If the survey 
is conducted in a way that systematically favours the inclusion of some 
groups at the expense of others then, again, a bias will result. If, for example, 
the survey involves researchers calling at home addresses between 9 a.m. 
and 5 p.m., this will tend to miss contact with those who are at work. To 
avoid this they would need to make contact in the evenings as well as during 
the day. 
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Good practice: avoiding non-response bias 


Researchers should take what practical measures are possible to find out if 
respondents are different from non-respondents in some way that is relevant to 
the survey and which might result in a non-response bias. 


Non-response bias and Internet surveys 


As with postal surveys the response rate from Internet surveys is influenced by 
the visual appeal of the questionnaire and the ease with which it can be 
answered and returned. Likewise, the response rate is boosted where potential 
respondents are contacted in advance, especially where the contact involves a 
personalized form of message in which people are greeted by name. And a 
planned follow-up of non-responses enhances the response rate just as it does 
with a postal survey. However, in terms of response rates and refusals to par- 
ticipate, fears have been voiced in connection with the use of Internet surveys. 
It has been suggested that if people feel ill at ease using computers or the 
Internet, the chances are that they will be reluctant to participate in online 
research. It would appear, though, that such anxieties are largely unfounded. 
The evidence would point to the fact that postal surveys and Internet-based 
questionnaires produce similar response rates and, if anything, there are indi- 
cations that online research might actually be preferred where either is an 
option (Truell et al. 2002; Dillman 2007). 

Bias through non-contact, however, remains a significant issue for Internet 
research. Internet research that relies on responses from visitors to websites has 
the problem that it cannot presume that those who visit the site constitute 
a cross-section of the population. The issue relates to the use of chat rooms, 
news groups and online interviews as well. The concern is that those who are 
contacted and who respond might be representative, not of the general public 
but just of those who are online and interested in a particular subject. 

There is also the matter of Internet connectivity. Although it is often stated that 
the Internet eliminates the boundaries of space and opens up the prospect of 
data collection on a global scale, the reality is that certain regions of the world 
are more likely to be online than others. Since the vast majority of Internet 
users live in the USA, Europe or Asia Pacific rim countries, those who live in 
poorer parts of the world are less likely to be able to participate in Internet 
research. And, although a large and increasing proportion of the population 
in developed countries is online, there are still demographic differences that 
need to be taken into consideration when it comes to who uses the Internet 
the most. The situation could well change rapidly and might vary from coun- 
try to country but, in the UK in the early years of the new millennium: 


e access to the Internet tends to correlate with social class, with wealthier 
groups being more likely to have access than poorer groups; 
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e men are more likely to use the Internet than women; 
e the elderly are less likely to have access to the Internet than other groups. 


Acceptable response rates 


There is no hard and fast rule about what constitutes an acceptable response 
rate. With large-scale postal questionnaire surveys, for instance, it will not be 
uncommon to get a response rate as low as 10 per cent. Face-to-face interviews, 
arranged by personal contact between the researcher and the interviewees, are 
the kind of approach at the other end of the spectrum where very high 
response rates can be expected — possibly even 100 per cent. Rather than look 
for a figure above which a response rate is acceptable and below which the 
results become suspect, it is more productive to evaluate the response rate 
that is actually achieved in terms of the following questions. 


e Is the level of response reasonable and in line with comparable surveys? 
The researcher can look to similar studies as a way of gauging whether the 
response rate is acceptable. The methods, the target group, the topic of 
research, the sponsor of the research and the use of prior contact are all 
important factors here. Each has a bearing on the level of response. The 
benchmark, then, needs to be set by the experience of similar surveys. 

e Have appropriate measures been taken to minimize non-response rates? 
There are practical measures which can be taken to encourage good 
response rates and to keep non-responses to the lowest possible level. 
Things that annoy respondents and give them cause not to reply should be 
anticipated ahead of the survey itself, and steps should be taken to elimin- 
ate them. Steps should also be taken to ‘follow up’ non-responses. The 
researcher should always build in some tactics for re-contacting anyone 
who does not respond on the first occasion, and be prepared to repeat the 
reminder two or three times if needed. 

e Do the non-respondents differ in any systematic and relevant fashion from those 
who have responded? 
Of course, to answer this question the researcher needs to have some infor- 
mation about those who have been targeted but not responded. This may 
prove difficult. None the less, it is good practice to endeavour to get some 
data about the non-respondents and to assess whether they are different 
from the respondents in any way that will have a bearing on the findings. 


SURVEYS AND SAMPLING 23 


Sampling 


The basic principle of sampling is that it is possible to produce accurate find- 
ings without the need to collect data from each and every member of a survey 
‘population’. For survey researchers this can be an attractive proposition. It 
means that they might be able to save time and money by reducing the 
amount of data they need to collect without, at the same time, reducing the 
accuracy of their findings. With large surveys, in particular, the benefits of 
sampling are easy to imagine. If the survey includes a sample of 1,000 people 
instead of all 100,000 in the group being studied, it will clearly take far less 
time and cost far less money. But the crucial point is that, in principle, such 
savings can be achieved without sacrificing the accuracy of the findings. 


Populations and samples 


Before looking at the approaches to sampling that a survey researcher can use, 
it is important to be clear about the terms ‘population’ and ‘sample’. In the 
context of surveys and sampling, the term ‘population’ has a specific meaning 
(Figure 1.1). Rather than meaning everyone who lives in a country, it refers to 
all the items in the category of things that are being researched. It means a 
research population. So, if a survey is used to study nurses in Scotland, then the 
survey population refers to all nurses working in Scotland (not all the people 
who populate the country) and the sample refers to those nurses who are 
selected to take part in the research. Likewise, if the research involves a survey 
of trade union members in the transport industry in Wales, then ‘population’ 
refers to all the transport workers in Wales who belong to a trade union, and 
the sample refers to the relatively small part of that (research) population who 
are actually chosen to participate in the study. 


FIGURE 1.1 Population and sample 


POPULATION 
all members of the 
group being studied 


SAMPLE 
small-subgroup 
chosen for study 
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Representative samples and exploratory samples 


When survey researchers select a sample from a population, they can be trying 
to do one of two things. They can be trying to get a representative sample or 
they can be trying to get an exploratory sample. Each involves a different 
approach and they tend to be associated with different kinds of social research. 

Representative samples tend to be associated with larger surveys and with the 
use of quantitative data. A representative sample involves a cross-section of 
the population. It matches the population in terms of its mix of ingredients, 
and relies on using a selection procedure that: 


e includes all relevant factors/variables/events, and 
e matches the proportions in the overall population. 


Information gathered from a representative sample allows the researcher to 
draw valid conclusions about how things are in the overall research population. 

Exploratory samples are often used in small-scale research and tend to lend 
themselves to the use of qualitative data. An exploratory sample is used as a 
way of probing relatively unexplored topics and as a route to the discovery of 
new ideas or theories. The point of the sample is to provide the researcher with 
a means for generating insights and information. For this purpose it is not 
always necessary to select people/items for the sample in terms of getting an 
accurate cross-section of the population. There are times when this might be 
the case, but selection to an exploratory sample is based principally on the 
need to gather new insights. Rather than go for typical examples, the explora- 
tory sample is more likely to include extreme examples or unusual examples 
of the thing being studied. 


Good practice: choosing a representative or exploratory sample 


Researchers need to be very clear and explicit about whether their research is 
based on a representative sample or an exploratory sample. 


Probability sampling and non-probability sampling 


There are two approaches to the selection of samples. Researchers can either 
choose probability sampling as the basis for selecting their sample from the 
research population, or they can use non-probability sampling. 

Probability sampling relies on the use of random selection. It is known as 
probability sampling because it is based on statistical theory relating to the 
‘normal distribution’ of events. The theory behind its use is that the best way 
to get a representative sample is to ensure that the researcher has absolutely no 
influence on the selection of people/items to be included in the sample. The 
sample, instead, should be based on completely random selection from the 
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population being studied. Probability sampling works best with large 
numbers, where there is a known population (number, characteristics) and it 
tends to be associated with large-scale surveys using quantitative data. 

Non-probability approaches to sampling do not operate on the principle of 
random selection to the sample and are used when researchers find it difficult 
or undesirable to choose their sample on the basis of pure chance. The reasons 
for this are varied but, in the main, they will be because: 


e The researcher feels it is not feasible to include a sufficiently large number 
of examples in the study. 

e The researcher does not have sufficient information about the population 
to undertake probability sampling. The researcher may not know who, or 
how many people or events, make up the population. 

e It may prove exceedingly difficult to contact a sample selected through 
conventional probability sampling techniques. For example, research on 
drug addicts or the homeless would not lend itself to normal forms of prob- 
ability sampling. 


Non-probability sampling involves an element of discretion or choice on the 
part of the researcher at some point in the selection process. Non-probability 
sampling can still retain the aim of generating a representative sample. In the 
interests of saving costs, however, the selection of the sample involves an 
element of expediency and established good practice rather than strict adher- 
ence to the principles of random selection. Non-probability sampling can 
also be used where the aim is to produce an exploratory sample rather than a 
representative cross-section of the population. Such approaches involve 
non-probability sampling because people or items are selected to the sample 
on the basis of things like their expertise, their experience or the fact that they 
might be unusual or different from the norm; their selection is not a matter of 
pure chance. 


Sampling frames 


A sampling frame contains information about the research population. Gener- 
ally it takes the form of a list of names or addresses that includes all of the 
members of the population from which the sample is to be selected. Some- 
times a suitable sampling frame can be easy to find. Research that involves 
formal organizations can often draw on things like membership lists, email 
addresses, and employee records to provide a suitable sampling frame that 
includes all members of the population to be researched. The information 
is already kept for other administrative purposes. Research involving particular 
industries and occupations can find that things like trade directories, com- 
mercial listings and memberships of professional associations provide suitable 
sampling frames. And, surveys targeting the general public might be able 
to use electoral registers or the Postcode Address File (PAF) — a list of 
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postcodes and postal addresses produced by the Post Office which contains 
about 99 per cent of all private residential addresses. There can be costs associ- 
ated with some listings. Commercial companies will supply lists of businesses, 
schools, hospitals, charities and other organizations, but will charge for the 
service. However, it is possible to be quite specific about which types of organ- 
ization and which areas are to be included in the list. This suits the needs of 
small-scale research projects because the lists can be fairly small and supplied 
at relatively low cost. 


Internet surveys and sampling frames 


In general, the nature of the Internet makes good sampling frames relatively 
more difficult to find. This is because addresses, identities and contact points 
on the Internet are very different from postal addresses and phone numbers in 
the ‘real’ world. In cyberspace, an address and an identity can be very tempor- 
ary and provide no background information of value to the researcher. People 
often have more than one email address, use more than one Internet Service 
Provider (ISP) and there is no guarantee that any address will be used by just 
one individual. Having said this, there are certain types of email lists that 
are well suited to being used as a sampling frame when conducting Internet 
surveys. Primarily, this is when the research is focused on populations who 
exist in some sort of formal organization because the administration of organ- 
izations almost invariably requires the maintenance of good records of the 
membership, e.g. employees at work, pupils in schools, members of clubs, 
volunteers with charities. Such email lists tend to serve known populations 
and can therefore prove to be a valuable basis for a sampling frame. Other 
email lists exist which, while they allow the researcher to target particular 
groups, need to be treated more cautiously in terms of their potential use as a 
sampling frame. There are, for example, distribution lists (provided by Google, 
Yahoo!, Hotmail and others) that are predominantly based on special interest 
groups. These lists allow the researcher to target people with an interest in a 
particular topic or activity, but they might not necessarily include the entire 
research population and, in this sense, might not be complete. 


Bias in sampling frames 


There is a danger with the use of any sampling frame that it might be 
incomplete or out of date. A list of private addresses will not lead the researcher 
to those who are homeless and live on the streets. An electoral register will not 
include those under 18 years of age or those who, for whatever reason, have 
not registered to vote. Things omitted might well be different in some import- 
ant respects from the things which actually exist on the list, and overlooking 
them as part of the research can lead to a biased sample. Just as damaging for 
the purposes of research, registers can include certain items that should not 
be there. They can contain names of people who have passed on - in either a 
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geographical or a mortal sense. People move home, people die, and unless 
the register is regularly and carefully updated on a routine basis, there is the 
likelihood that it will be out of date and include items it should not. Either 
way, the impact can be significant. If the sampling frame systematically 
excludes things that should be in, or systematically includes things which 
should be out, the sample will almost inevitably be biased. 


Good practice: sampling frames 
A good sampling frame should be: 


e Relevant: it should contain things directly linked to the research topic. 

e Complete: it should cover all relevant items. 

e Precise: it should exclude all the items that are not relevant. 

e Up-to-date: it should incorporate recent additions and changes, and have 
redundant items cleansed from the list. 


Any deficiencies ought to be acknowledged. Good research does not depend 
on trying to hide limitations in matters like this. It is far better to recognize any 
ways in which the sampling frame might be less than perfect and to discuss 
the possible impact on the nature of the findings from the research. 


Probability sampling techniques 


Random sampling (Each person/item has an equal chance of being selected) 


Random sampling is often portrayed as the ideal basis for generating a repre- 
sentative sample. This is because with random sampling the inclusion of a 
person/item in the sample is based entirely on chance. Random selection 
ensures that there is no scope for the researcher to influence the sample in 
some way that will introduce bias. 

It is important to appreciate that the term ‘random’, as used in this context, 
does not mean that selection can be haphazard or based on an ‘anyone will 
do’ approach. And it is a dangerous mistake for the project researcher to use 
the term ‘random’ in a loose or colloquial way when dealing with sampling 
techniques. For research purposes, the term ‘random’ actually involves quite 
specific constraints on the way that persons/items are selected. From a stat- 
istical point of view randomness means that ‘each unit should have an equal 
and known probability of inclusion in the sample’. 

To get a random sample, there needs to be: 


1 A known population from which the sample is to be taken. The researcher 
needs to know something about the make-up of the overall population 
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from which the sample is to be extracted (such as the total number, who 
they are or where they exist). 

2 a sampling frame that provides a list of all the items in the population and 
which allows them to be picked via some unique identifier (such as a name, 
an address, an employee number or a birth date). 

3 a process of random selection which provides the basis for including 
specific units in the sample. For research purposes this can involve the use 
of a random number generator — a utility available free online or through 
appropriate computer software packages. 


Example: random sampling 


In a local football supporters club a random sample could be selected by 
putting all the members’ names (the ‘population’) into a large bucket and 
then, having ensured that the tickets are well mixed, getting an impartial and 
possibly blindfolded person to pick a set of names from the bucket. If asample 
of 100 is sought, then it is the first 100 to be picked who will constitute the 
sample. A random sample, then, is like a lottery draw. National lotteries oper- 
ate on the basis of selecting numbers at random, though in this case they use 
special machines to do the selection. The underlying principle in either case 
is that no person and no thing can influence which items get picked. The 
selection is based on just one factor — pure chance. 


Systematic sampling (Every nth case is included) 


Systematic sampling adheres closely to the principle of random selection but, 
in this case, the researcher selects items in a systematic way — picking every nth 
item from an appropriate list. For a sample of 50 from a population of 250 it will 
be every 5th item, for a sample of 70 from a population of 490 it will be every 
7th item - nth depending on the size of the required sample relative to the 
number in the research population. It should not matter where on the list the 
selection starts. The researcher, however, will want to feel sure that there is no 
order or pattern in the list that might be related to the topic being investigated 
because this could undermine the principle of random selection and cause 
some bias to creep into the sample as a result of the system of selection. 


Example: systematic sampling 


To follow the example of the local football supporters club, rather than put all 
the members’ names into a bucket the sample could be chosen from a printed 
list of the club’s members. From the list of 1,000 names, the selection of the 
100 members to be included in the sample would be based on choosing every 
10th person. Since, in this example, the membership list is based on the 
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alphabetical order of surnames there might seem no reason to doubt that the 
eventual sample will constitute a random sample, just as it would have done 
had the names been picked out of the bucket. 


Cluster sampling (Clusters within the population) 


The question of resources needs to be taken seriously when it comes to the 
selection of samples. Identifying units to be included, contacting relevant 
respondents and travelling to locations can all entail considerable time and 
expense. The virtues of a purely random selection, then, might need to be 
weighed against the savings to be made by using alternative approaches 
which, while they retain a commitment to the principles of random selection 
and the laws of probability, try to do so in cost-effective ways. 

Cluster sampling is a typical example of this. Rather than using random 
sampling to select individual people or items, it uses random sampling to 
select specific clusters or groups. Within each of the clusters all people/items 
are included in the sample. The advantage of this sampling technique is that 
the clusters, as the name suggests, contain items that are closely grouped 
together — normally in one location or in one geographical area. By focusing 
on such clusters, the researcher can save a great deal of time and money that 
would otherwise have been spent on travelling to and fro visiting research 
sites scattered throughout the length and breadth of the land. For this reason 
cluster sampling is often used by market researchers and others who need to 
conduct surveys on a limited budget. 

There are, however, two important conditions that need to be met for the 
use of cluster sampling. First, the clusters must be pre-existing, naturally occur- 
ring groups. Second, each cluster should reflect the heterogeneity of the total 
population. The idea is that within each cluster the researcher might expect to 
find a mini-version of the total population and that by including all members 
of the cluster within the sample the researcher will be able to get a representa- 
tive sample that will produce findings that accurately match the situation in 
the total research population. 


Example: cluster sampling 


A cluster sample could be based on a random selection of schools and the 
inclusion of all students within those schools. Schools are naturally occurring 
units in the sense that the researcher does not need to organize the grouping of 
all the young people on one site — they are there anyway. The inclusion of all 
students within the selected schools is based on the assumption that within 
each cluster there will be a cross-section of the wider population in terms of 
things like age, sex, ethnicity, social background and academic ability. 
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Multi-stage sampling (Samples drawn from samples) 


Multi-stage sampling involves selecting samples in a sequence of stages, each 
sample being drawn from within the previously selected sample. The under- 
lying principle is still one of random selection, only in this case it involves 
sampling at multiple stages rather than all at once. In principle, multi-stage 
sampling can go on through any number of levels, each level involving 
a sample drawn from the previous level. 

Multi-stage sampling is often associated with cluster sampling. The distinc- 
tion between the two is that cluster sampling uses all the items or people that 
exist within each cluster whereas multi-stage sampling selects a sample from 
within the cluster. This is particularly useful if: (1) the researcher suspects that 
the population within the chosen clusters will not provide a cross-section of 
the wider population or (2) if resources dictate that it is too expensive to 
include everyone in the cluster. 


Example: multi-stage sampling 


From a research population of 5,000 schools researchers could randomly 
select a sample of 50 schools for inclusion in their study. Within each of the 
50 schools they could then proceed to randomly select 40 students — providing 
an overall sample of 2,000 students on which to conduct a survey. Sub- 
sequently, a small random sample of 100 students might be selected for 
further study — 1 in 20 selected at random from the sample of 2,000 students. 
As with cluster sampling this would be administratively more convenient 
than simple random sampling because samples would be taken from just 
50 locations. 


Stratified sampling (Sampling based on subgroups in the population) 


Stratified sampling continues to adhere to the underlying principle of random 
selection. However, it introduces some element of researcher influence into 
the selection process and, to this extent, moves away slightly from pure ran- 
dom sampling. Stratified sampling subdivides the research population into 
different subgroups (strata) and then chooses the required number of items or 
people from within each subgroup using random sampling techniques. The 
reason for introducing stratification into the selection process is to ensure that 
crucial parts of the population are appropriately represented in the overall 
sample. It allows the researcher to select a sample which he/she knows will 
include, for instance, equal number of men and women, or perhaps an appro- 
priate balance of different age groups within the sample. There are two 
important conditions that apply to the use of stratified sampling. First, the 
subgroups must be fairly homogenous in the sense that they share something 
in common which can be used to separate them into clearly identifiable 
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subgroups. Second, this factor must be relevant and significant in terms of 
what is already known about the topic. 


Example: stratified sampling 


A researcher who wishes to collect information about voting behaviour will 
know in advance, from demographic data, that the population of voters 
from whom data are to be collected will include a given proportion of 
males and females, and will include given proportions of different age bands 
from 18 years up. The researcher should also realize from a review of the 
literature that sex and age are factors linked with voting behaviour. When 
constructing the sample, then, the researcher could wisely choose to adopt a 
stratified sampling approach in which: 


e all relevant categories of sex and age are included; 
e the numbers included for each category are directly in proportion to those 
in the wider population (all voters). 


In this way, the voting intentions displayed by the sample are likely to corres- 
pond with the voting intentions in the wider population of voters. 


An illustration of probability sampling techniques 


Table 1.1 serves to illustrate how different probability sampling techniques 
can operate. The numbers involved are small which, for the purposes of an 
illustration, is actually quite useful because it highlights some of the problems 
that can arise when probability sampling is used with small sample sizes. 
In practice, however, the size of the samples would need to be considerably 
larger to allow the probability samples to produce sufficiently accurate results. 
Table 1.1 does not include multi-stage sampling simply for reasons of visual 
clarity. 

In this illustration the population consists of 27 people. Let us presume 
that they are members of a football supporters club and that we wish to select 
a sample of 9 people to take part in our research (and bear in mind that, in 
reality, the numbers might be one hundred times greater than this miniature 
illustration). The sampling frame consists of a list of members’ names. These 
are ordered alphabetically, and each name has been assigned a number. In 
principle, this is sufficient for us to produce either a pure random sample or 
a systematic random sample. The sampling frame contains two further bits 
of information about the members — location and sex. These permit the use of 
cluster sampling and stratified sampling. 

A random sample can be selected using random number generating soft- 
ware. Simple utilities for this can be found on the Internet, or they come as 
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part of standard data analysis software. We use the software to generate a list 
of 9 numbers between values 1 and 27. Those whose names are matched 
with these numbers on the list are then selected to form the random sample. 
Note that in this particular list there are 7 males and 2 females. Running 
the random number generator again will produce different figures and, in all 
probability, a different balance of males to females. This effect is exaggerated 
by the very small numbers involved but as the sample size increases towards 
1,000 or more the difference caused by using different random numbers 
would vastly diminish. 

A systematic sample can be produced simply by selecting every third person 
on the list. This will provide the researcher with 9 people who have been 
selected simply by chance in terms of where they happen to appear on the list. 
This presumes, of course, that there is nothing in the order of the list that 
could bias the sample. Note that in this particular systematic sample there are 
9 males and O females. However, if the start point is changed, the proportions 
change dramatically. Check how the balance changes when we start the 
systematic selection at number 001 (3 males, 6 females), or number 002 
(6 males, 3 females) instead. 

For a cluster sample the selection is based on the random selection of 
3 clusters and the inclusion of all those within the cluster as part of the 
sample. In this illustration the cluster is based on location. If locations B, H 
and | happen to be randomly selected, the sample will incorporate the 9 
people who exist in these locations. The assumption here is that location itself 
is not a significant factor that could bias the sample. 

A stratified sample is designed to ensure that there is an appropriate repre- 
sentation of the population within the overall sample. In this illustration a 
stratified sample could be used to ensure that males and females are repre- 
sented proportionately in the sample. We know from our sampling frame that 
there is a ratio of 2:1 of males to females in our population (the football 
supporters club members). To get a stratified sample we need to segregate the 
membership list according to sex. This is how the sampling frame is reorgan- 
ized on the right of Table 1.1 — males separated from females. Then, 6 random 
numbers can be generated for selecting from the list of males and 3 random 
numbers likewise for selecting from the list of females. In this way the 
researcher guarantees to get a sample where appropriate proportions have 
been randomly selected from within specific strata of the membership of the 
football supporters club. 
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Non-probability sampling techniques 


Quota sampling (Selected to meet specific criteria) 


Quota sampling is widely used in market research. It operates on very similar 
principles to stratified sampling. It establishes certain categories (or strata) 
which are considered to be vital for inclusion in the sample, and also seeks to 
fill these categories in proportion to their existence in the population. There is, 
though, one distinctive difference between stratified and quota sampling. 
With quota sampling, the method of choosing the people or events that make 
up the required number within each category is not a matter of strict random 
selection. In effect, it is left up to the researcher to choose who fills the quota. 
It might be on a ‘first to hand’ basis - as when market researchers stop 
people in the street. The people might be appropriate but were chosen because 
they just happened to be there, not as part of a random selection from a 
known population. The technical difference here excites statisticians but need 
not trouble the project researcher too much. The crucial point is that, like 
stratified sampling, it has the advantage of ensuring the representation of 
all crucial categories in the sample in proportion to their existence in the 
wider population. It does so without waste. Because the quotas are set in 
advance, no people or events that subsequently become ‘surplus to require- 
ments’ are incorporated into the research. Quota sampling, then, has particu- 
lar advantages when it comes to costs — especially when used with face-to-face 
interviewing. 


Example: quota sampling 


Stores in a shopping mall might want to know what their customers think 
about the facilities that are available (e.g. parking, toilets, restaurants) and 
whether there are any particular features of the mall that attract them to visit 
it. A quota sample for face-to-face interviews in the mall could be devised on 
the basis of including 200 males and 200 females, and within each group there 
could be quota of 40 in each of five age categories 12-18, 19-30, 31-45, 
46-64, 65+. The quotas for interview, then, would be 40 males aged 12-18, 
40 females aged 12-18, 40 males aged 19-30, 40 females aged 19-30, and 
so on. Interviewers would approach appropriate shoppers and continue to 
conduct interviews until they had filled the quota for each subgroup. 


Purposive sampling (Hand-picked for the topic) 


Purposive sampling operates on the principle that we can get the best informa- 
tion through focusing on a relatively small number of instances deliberately 
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selected on the basis of their known attributes (i.e. not through random 
selection). With purposive sampling the sample is ‘hand-picked’ for the 
research on the basis of: 


e relevance: to the issue/theory being investigated; 
e knowledge: privileged knowledge or experience about the topic. 


Purposive sampling works where the researcher already knows something 
about the specific people or events and deliberately selects particular ones 
because they are seen as instances that are likely to produce the most valuable 
data. They are selected with a specific purpose in mind, and that purpose 
reflects the particular qualities of the people or events chosen and their rele- 
vance to the topic of the investigation. 

A purposive sample can be used in order to ensure that a wide cross-section 
of items or people is included in the sample. When used in this way, purpos- 
ive sampling is, to a degree, emulating a ‘representative sample’. Based on prior 
knowledge, the researcher can deliberately select the sample in order to ensure 
that the full range of items or people get included (e.g. all age-groups, all 
ethnic groups). This can be useful with small-scale surveys where random 
sampling of itself might not be likely enough to include groups that occur in 
relatively small numbers in the population. 


Example: purposive sampling for a cross-section of the population 


A purposive sample might deliberately include members of the population 
aged over 90 years, a group that might get overlooked using random sampling 
with relatively small samples simply because they are relatively small in num- 
ber compared with other age-groups. 


Purposive sampling can also be used as a way of getting the best information 
by selecting items or people most likely to have the experience or expertise to 
provide quality information and valuable insights on the research topic. Used 
in this way, purposive sampling is particularly well suited for creating an 
‘exploratory sample’. The advantage of purposive sampling is that it allows the 
researcher to home in on people or events which there are good grounds for 
believing they will be critical for the research. 

Instead of going for the typical instances, a cross-section or a balanced 
choice, the researcher can concentrate on instances which will best illuminate 
the research question at hand. These might be extreme cases, rare cases or 
critical cases rather than ones chosen to be representative. 
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Miles and Huberman (1994) call these special instances ‘outliers’, and com- 
mend their inclusion in the process of discovery as a check which explores 
rival possible explanations and tests any explanation based on mainstream 
findings, by seeing if they can work with instances which are distinctly not 
mainstream. 


Example: purposive sampling for key informants 


A study of online banking fraud might start by identifying some individuals 
who play key roles in the detection and prevention of such fraud. Major high 
street banks have specialist teams dedicated to online security and the 
researcher could make contact with these teams to arrange interviews. Victims 
of online banking fraud could be sought to provide their particular perspective, 
and the researcher might even try to include fraudsters themselves to get even 
more insight into the topic. 


Theoretical sampling (Selected to help generate theories) 


With theoretical sampling, the selection of instances follows a route of dis- 
covery based on the development of a theory which is ‘grounded’ in evidence. 
At each stage, new evidence is used to modify or confirm a ‘theory’, which 
then points to an appropriate choice of instances for research in the next 
phase. The sample evolves, and it continues to grow until such time as the 
researcher has sufficient information in relation to the theory that is being 
developed. 


Link up with Grounded theory, Chapter 6 A 


Example: theoretical sampling 


A study of teachers’ control of classrooms in secondary schools might involve 
observation in classrooms and depth interviews with teachers. From early 
fieldwork the researcher could be alerted to the particular significance of noise 
emanating from classrooms as an indicator of (lack of) control. Building on 
this, theoretical sampling would involve choosing further observations and 
interviews deliberately to make comparisons and to check the initial ‘theory’ — 
perhaps looking for instances where noise might not be likely to signify a lack 
of control (e.g. music lessons or PE lessons). 
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Snowball sampling (Participants refer the researcher on to other 
potential participants) 


With snowball sampling the sample emerges through a process of reference 
from one person to the next. At the start, the research is likely to involve just 
one or a few people. Each can be asked to nominate some other people who 
would be relevant for the purposes of the research. These nominations are 
then contacted and, it is hoped, included in the sample. The sample thus 
snowballs in size as each of the nominees is asked, in turn, to nominate further 
persons who might be included in the sample. 

Snowball sampling is an effective technique for building up a reasonable- 
sized sample, especially when used as part of a small-scale research project. 
One advantage is that the accumulation of numbers is quite quick, using the 
multiplier effect of one person nominating two or more others. Added to this, 
the researcher can approach each new person, having been, in a sense, spon- 
sored by the person who had named him or her. The researcher can use the 
nominator as some kind of reference to enhance his or her bona fides and 
credibility, rather than approach the new person cold. Snowball sampling is 
particularly useful where there is no sampling frame of any kind which can 
allow the researcher to identify and make contact with appropriate partici- 
pants. For this reason, it is often used in conjunction with qualitative research 
based on small-scale exploratory samples, and its use is perfectly compatible 
with purposive sampling and theoretical sampling. People can be asked to 
nominate others who meet certain criteria for choice, certain conditions related 
to the research project and certain characteristics such as age, sex, ethnicity, 
qualifications, residence, state of health or leisure pursuits. 


Example: snowball sampling 


A study of smoking cessation among ‘underage’ smokers could use snowball 
sampling to good advantage. There is no sampling frame for such people and 
there is no obvious or simple way to identify such people. So, the researcher 
could make enquiries until one such person is found. This person is likely to 
know of others his/her age who have started smoking and subsequently quit 
before they reached the age of 16. Assuming the young person is willing to 
co-operate, he/she could be asked to identify two others who have quit smok- 
ing. In turn, these two could be asked to identify others who fit the needs of the 
sample, and so on. This process builds a sample that snowballs in size. 


Convenience sampling (First to hand) 


Convenience sampling is built upon selections which suit the convenience 
of the researcher and which are ‘first to hand’. An element of convenience is 
likely to enter into sampling procedures of most research. Because researchers 
have limited money and limited time at their disposal, it is quite reasonable that 
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where there is scope for choice between two or more equally valid possibilities 
for inclusion in the sample, the researcher should choose the most conveni- 
ent. If, for example, two or more clusters are equally suitable as research sites, it 
would be crazy to opt for ones that were the furthest away without some good 
reason to do so. Convenience sampling, however, takes this to an extreme by 
using convenience as the main basis for selecting the sample, not a subsidiary 
one. The criterion for selection to the sample is that items are convenient and 
the key advantage of convenience sampling is that it is quick, cheap and easy. 


Example: convenience sampling 


A student who wishes to conduct a survey into consumer preferences in rela- 
tion to soft drinks might choose all the fellow students on the course to be the 
‘sample’. They are easy to find and can probably be persuaded fairly easily 
to complete a questionnaire on the subject. Alternatively, using face-to-face 
interviews, the student could collect data from students using the canteen. 
Those who are interviewed will constitute a ‘convenience sample’. 


A Caution 


Convenience itself offers nothing by way of justification for the inclu- 
sion of people or events in the sample. It might be a reasonable prac- 
tical criterion to apply when faced with equally viable alternatives but, 
in its own right, is not a factor that should be used by researchers to 
select the sample. Choosing things on the basis of convenience runs 
counter to the rigour of scientific research. It suggests a lazy approach 
to the work. Good research selects its items for study not on the basis 
that they are the easiest to obtain but for specific reasons linked to the 
subject matter of the research and the requirements of the investiga- 
tion. For this reason, the practice of convenience sampling is hard to 
equate with good research. 


Which sampling technique should be used? 


From the outline of sampling techniques above we can see that the survey 
researcher has 10 basic options from which to choose, each characterized by a 
different method of selecting the people or items to be included in the sample. 
As Figure 1.2 shows, the sampling techniques sit somewhere on a continuum 
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with, at one extreme, a selection technique which eliminates altogether the 
influence of the researcher (random selection) and at the other extreme, a 
selection technique where selection of the sample is left entirely to the discre- 
tion of the researcher (convenience sampling). In between, there are a range of 
techniques which differ in the extent to which they adhere to the principle of 
random selection. But not only do the techniques sit on a sliding scale in terms 
of the basis for selecting the sample, they also have corresponding positions 
in terms of the purpose of the sample, the kind of data to be collected, the size 
of the sample and the amount of resources they are liable to consume. The 
choice of sampling technique depends, therefore, on how the various tech- 
niques fit the needs of the researcher and the kind of research he/she proposes 
to undertake. When deciding which technique to use, the researcher needs to 
consider five core questions: 


1 Is my purpose to produce a representative sample or an exploratory sample? 

2 To what extent is this purpose better served by selecting the sample on this 
basis of random selection or deliberate choice? 

3 Will qualitative data or quantitative data be more appropriate for addressing 
my research questions? 

4 Does a suitable sampling frame exist and can I gain access to it? 

5 In terms of the resources available for the research (time and money) 
which sampling technique is both feasible and likely to produce relevant 
information? 


Size of the sample 


There are basically three approaches to the calculation of the sample size: 
statistical, pragmatic and cumulative. The statistical approach is usually pre- 
sented as the proper approach. As we shall see, however, it is best suited to 
large-scale surveys and probability sampling techniques. It is the kind of 
approach that works well for things like opinion polls and government surveys 
which involve very large populations and which cost a lot of money. Smaller- 
scale surveys tend to use a more pragmatic approach. This is partly to do 
with the costs and partly to do with the difficulties in meeting all the condi- 
tions needed for the statistical approach to sample-size calculation. Market 
researchers often use a pragmatic approach, estimating the sample size on the 
basis of years of practical experience and on what works well enough within 
given resource constraints. The third approach is normally associated with 
small-scale, qualitative research. The cumulative approach is one in which the 
researcher continues to add to the size of the sample until a point is reached 
where there is sufficient information and where no benefit is derived from 
adding any more to the sample. 
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The purpose of the sample: representative or exploratory 


When deciding which approach to take, the researcher needs to bear in mind 
the kind of sample that is required. Where the aim is to get a representative 
sample, then the numbers involved will tend to be relatively large, and where 
the aim is to produce an exploratory sample, the numbers involved will be 
relatively small. There are different considerations for either kind of sample 
and any discussion of sample size needs to reflect the different premises on 
which they operate. 

In the case of representative samples there is something of a ‘big is beautiful’ 
stance on survey size. This is based on the principle that the more instances 
that are covered, the less likely it is that the sample will be biased. With a large 
sample, the researcher is more assured that: 


e all aspects of relevance to the research question will have been covered and 
included in the findings; 

e there will be some balance between the proportions within the sample and 
the proportions which occur in the overall population being investigated. 


Both factors enhance the representativeness of the sample and, in turn, this 
allows greater confidence about making generalizations based on findings 
from the sample. 

With exploratory samples the scale of the research tends to be smaller. There 
are two main reasons for this. First, it is likely that each item or person to be 
included in the sample will be studied in greater depth and more detail than 
will be the case with a representative sample. There is no absolute rule on this, 
but the size of exploratory samples generally reflects the fact that researchers 
want to probe deeper than they would do with representative samples. Second, 
the size of exploratory samples is not governed by matters of accuracy but by 
considerations of how informative the sample is. This means that the sample 
only needs to be sufficient in size to enable the researcher to feel that enough 
information has been collected. 


The anticipated response rate 


When thinking about sample size, there is an important distinction to be 
made between the sample size that the researcher eventually has available 
to use for data analysis and the sample size that is initially contacted in the 
survey. A survey rarely achieves a response from every contact. Especially when 
using postal questionnaires and the like, the rate of response from those con- 
tacted is likely to be pretty low. As far as sample size is concerned, though, the 
important thing for the researcher to consider is that the number in the original 
sample may not equal the number of responses that are finally obtained which 
can be used in the research. The researcher needs to predict the kind of 
response rate he or she is likely to achieve and build into the sample size an 
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allowance for non-responses. If the researcher wants to use a sample of some 
100 people for research and is using a postal questionnaire survey for which 
a response rate of 30 per cent is anticipated, the original sample size needs 
to be 334. 


The statistical approach (large-scale surveys using probability sampling 
for a representative sample) 


If the survey is large-scale and it uses probability sampling, there is a precise 
way of calculating the required sample size. This is based on statistical theory 
and the normal curve distribution of events. Without getting involved too 
heavily in statistical theory, the calculation of an appropriate sample size will 
depend on four things: 


e The size of the research population. It is important to appreciate, though, that 
population size is significant where smaller numbers are concerned but, as 
Table 1.2 indicates, when the population moves beyond 5,000, changes in 
the size of the population cease to have much impact on the sample size. 

e The accuracy of the estimates. Social researchers need to feel confident that 
their estimates lie close to the real (i.e. population) figure and they will 
normally tolerate a margin of error of 3 per cent or 5 per cent. More accurate 
results such as the 1 per cent margin of error require sample sizes that do 
not offer much saving until the population size exceeds 5,000. The margin 
of error that is deemed acceptable for a specific project is known as ‘the 
confidence interval’. 

e Confidence that the sample will produce results that are representative of 
the population. This is the level of confidence. Social researchers know that 


Table 1.2 Sample size and population size (probability sampling) 


Number in the Required sample size! (at 95% confidence level) 
population 
5% 3% 1% 
margin of error margin of error margin of error 
50 44 48 50 
100 80 92 99 
250 152 203 244 
500 217 341 475 
1,000 278 516 906 
5,000 357 879 3,288 
10,000 370 964 4,899 
100,000 383 1,056 8,763 
1 million 384 1,066 9,513 
10 million 384 1,067 9,595 


Note: 1 Refers to final usable responses. 
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there is always a chance that the sample might be extreme and unusual, 
and they want to feel very confident that their findings are not the product 
of any such extreme or unusual sample. In practice, they tend to want a 
95 per cent confidence level. They want to know that there would be a less 
than 1 in 20 chance of this occurring. 

e Variation in the population: less variation requires a bigger sample size. In 
terms of the particular attribute being measured (e.g. voting intention), it 
matters whether the population is evenly matched (nearing a 50:50 split) 
or is widely divided (e.g. 70:30 split). Researchers can get some idea of this 
in advance (i.e. before they actually conduct their survey) either from a 
review of existing research or by using a pilot survey. If this factor cannot be 
estimated, it should be assumed that the population is evenly matched 
because this is a ‘worst case scenario’ as far as things are concerned when it 
comes to the size of the required sample. By default, the software plays on 
the safe side. 


Taking these four factors into consideration it is possible to calculate the 
required sample size. There is a formula for this calculation but the maths do 
not need to concern the project researcher because there are plenty of software 
utilities freely available on the Internet that do the calculations at the press 
of a button. A web search on ‘sample size calculator’ will identify plenty of 
these. Some of these ask for variation (or ‘response distribution’) figures while 
others do not. If in doubt, enter 50% in this box. Where they do not ask the 
user for the variation they make the worst case scenario assumption outlined 
above. What all the sample size calculators do, however, is demonstrate very 
clearly how the required sample size varies according to what the researcher 
wants in terms of levels of confidence and margins of error. The greater the 
level of confidence and the greater the level of accuracy, the bigger the size of 
the sample. 


FIGURE 1.3 Sample size calculator 


Determine Sample Size 


Confidence Level 95% 99% 
Confidence Intervak 

Population: 

[ catewiste ] [Clear | 


Sample size needed: | 
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Accuracy and sample size 


To achieve greater accuracy, the researcher might need to increase the size of 
the sample. Statistical procedures can be used to calculate what specific sample 
size will be necessary in order to achieve a given level of accuracy. However, 
there is an interesting point that springs from statistical estimates and sample 
size. It is that there is relatively little advantage to be gained in terms of accur- 
acy once a sample has reached a given size. There are diminishing returns to 
increases in the size of samples. This runs contrary to common sense, which 
would probably say to us that the degree of accuracy of results would depend 
on what proportion of the population is included in the sample. Common 
sense might say that a sample of 25 per cent of cases will produce better results 
than a sample of 10 per cent. Statistics would say that where the population 
size is large, there is hardly any increase in accuracy to be obtained by 
incorporating another 15 per cent (see Table 1.2). 


Key point: sample size 

The crucial factor to be considered in relation to sample size is not the 
proportion of the population which is included in the survey, but the 
absolute size of the sample. 


Non-responses and sample size 


With probability sampling, the researcher should initially calculate the 
required sample size. Then, to achieve this sample size, the researcher needs 
to include sufficient numbers in the survey to allow for (1) the likely propor- 
tion of non-responses; and (2) the likely amount of returns that, for what- 
ever reason, cannot be used in the final dataset. With a research population 
of 10,000 and a confidence level of 95 per cent with a 5 per cent margin of 
error, an eventual sample size of 370 is required. On the basis of similar sur- 
veys, it might be calculated that there will be a response rate of 55 per cent and 
that non-usable responses will account for 5 per cent of all those that are 
returned. On this basis the researcher could work out that he/she needs to 
distribute the survey to 708 of the population. From the 708 included in the 
survey a 55 per cent response rate will lead to about 389 returns, and if 95 per 
cent of these are fully usable, then this will provide the 370 sample size that 
is required. 


Good practice: sample size and non-responses 


Always allow for the likely non-response rate when calculating the numbers 
needed in order to produce the required sample size. 
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/\ Caution 


There are a number of assumptions which underlie the statistical cal- 
culation of sample size. As Gorard (2006: 4) points out ‘The sample 
must be selected (or allocated) randomly from a known population... 
The sample must be complete, with full response and no respondent 
dropout ... The measurements involved must be made without error 
... and... the measurements involved are often expected to follow 
an ideal distribution, such as the normal, or Gaussian, curve.’ From a 
practical point of view it is very difficult for survey research to fully 
meet such conditions (Gorard 2006). This would suggest that the 
calculation of sample size using probability theory is best viewed as 
being more relevant to large-scale surveys operating under near-ideal 
conditions rather than something that can be applied directly to 
smaller-scale surveys. 


The pragmatic approach (smaller-scale surveys using non-probability 
sampling for a representative sample) 


In practice, social research frequently involves surveys with relatively small 
numbers — between 30 and 250 — and when estimating the required sample size 
such surveys tend to depend on non-probability sampling techniques. There 
are three pragmatic reasons for this. First, there is the matter of resources. In 
the real world of research the availability of resources is one factor that always 
affects the researcher’s decision on sample size. The fact is that research does 
not take place with infinite time and money. In practice, it has to be tailored 
to meet the constraints imposed by the amount of time and money which can 
be spent on the investigation, and there are not too many individuals or 
organizations who can actually afford the scale of investigation that involves 
samples of 1,000 or more. Project researchers such as those conducting a sur- 
vey as part of an academic degree are very unlikely to have the time or money 
to work with large populations and large samples. 

Second, there is the nature of research populations and the fact that many of 
the populations that social researchers might like to investigate are relatively 
small. Work organizations in the UK, for example, are predominantly small 
and medium-sized enterprises involving workforces of less than 250 people 
and any survey of individual workplaces might well be obliged to operate with 
relatively small numbers. And this is true of so many of the groups, events 
and locations the social researcher might want to study. Project researchers, in 
particular, are likely to want to focus on localized units involving smallish 
numbers of people or items. 


46 THE GOOD RESEARCH GUIDE 


Third, the pragmatic approach subscribes to the argument that, used prop- 
erly, non-probability sampling techniques can produce data that are suf- 
ficiently accurate for the purposes of research. Within the statistical approach, 
it should be noted, there are ways of accommodating the need for small sam- 
ples and coping with the likely impact of such small samples on the accuracy 
of the data they produce. The pragmatic approach, however, tends to rely on 
non-probability sampling instead. This is because it is less expensive to under- 
take research using things like quota sampling techniques rather than random 
sampling techniques. There is, however, an ongoing debate about whether the 
use of non-random sampling techniques with smaller sample sizes jeopardizes 
the accuracy of the data. The pragmatic approach takes the position that the 
level of accuracy needs to be weighed against the additional costs involved and 
that the aim is to get accuracy that is good enough for the purposes of research within 
the resources available for research. Commercial research companies actually 
advise potential customers that for a given sum of money they can be supplied 
with results within a given level of accuracy; a greater level of accuracy will cost 
more. As a consequence, the customer and the commercial research company 
need to agree about whether results will be accurate enough in relation to the 
resources available. 

The pragmatic approach, then, estimates sample sizes in relation to the scale 
of the investigation and a level of accuracy that is sufficient for the purposes of 
research. Using non-probability sampling, it does not rely on a statistical calcu- 
lation of the sample size but uses instead good judgement based on experience 
about what is feasible under the circumstances. 


$ 


Key point: sample size 


‘In practice, the complexity of the competing factors of resources and 
accuracy means that the decision on sample size tends to be based on 
experience and good judgement rather than relying on a strict 
mathematical formula.’ 

(Hoinville et al. 1985: 73) 


This good judgement takes into account four important factors. These do not 
provide the exact figure as would be the case with the statistical approach, but 
they do provide guidelines that can help the researcher decide which sample 
size might be appropriate and acceptable: 


e Comparison with other similar surveys. The researcher’s literature review 
should identify instances where other researchers have conducted surveys 
under comparable conditions, and the sample sizes used in such surveys 
can provide some starting point for deciding what numbers to include in 
the sample. 

e Samples should not involve fewer than 30 people or items. Certainly, it is a 
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mistake to use statistical analyses on samples of fewer than 30 without 
exceptional care about the procedures involved. 

e Allowance for the number of subdivisions likely to be made within the data. 
When calculating the number of people or events to include in the sample 
the researcher needs to take into consideration the complexity of the data 
that are likely to emerge. A sample size which initially looks quite large 
might produce only very small returns in relation to specific subdivisions. 
So, for example, a sample of 100 people used to investigate earnings and 
occupational status might need to be subdivided according to the age, sex, 
ethnicity, marital status and qualifications of the people, and according to 
whether they are full-time, part-time, unemployed, child-rearing or retired. 
This simple investigation would need a cross-tabulation of five personal 
factors by five occupational factors, i.e. 25 subdivisions of the data. If the 
data were equally distributed, this means that there would be only four 
cases in each of the subdivisions, which is hardly an adequate basis for 
making generalizations. In practice, of course, we know that the data would 
not be evenly distributed and that many of the subdivisions would end up 
with no cases in them at all. The researcher therefore needs to think ahead 
when planning the size of the sample to ensure that the subdivisions 
entailed in the analysis are adequately catered for. The smaller the sample, 
the simpler the analysis should be, in the sense that the data should be 
subjected to fewer subdivisions. 

e Awareness of the limitations. Special caution is needed about the extent to 
which generalizations can be made on the basis of the research findings. 
Provided that the limitations are acknowledged and taken into account, the 
limited size of the sample need not invalidate the findings. 


Good practice: sample size 
There is a general tendency to choose the minimum sample size that is 
feasible with respect to the available resources and the level of accuracy 
demanded of the findings. 


The cumulative approach (small-scale surveys using non-probability 
sampling for an exploratory sample) 


There are some approaches to social research where the size of the sample 
cannot be stated with certainty at the start of the investigation. The sample 
size grows during the course of the research and continues to grow until the 
researcher has accumulated sufficient information for the purposes of the 
research. This cumulative approach relies on non-probability techniques such 
as purposive, theoretical or snowball sampling and is normally associated with 
research that: 
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e is relatively small-scale; 

e uses qualitative data; 

e cannot identify the research population in advance; 

e aims to produce an exploratory sample, not a representative one. 


The cumulative approach to sample size is illustrated perhaps most neatly in 
the grounded theory approach and its use of theoretical sampling. Here the 
process is one of ‘discovery’. Almost like a detective, the researcher follows a 
trail of clues. As each clue is followed up, it points the researcher in a particular 
direction and throws up new questions that need to be answered. Sometimes 
the clues can lead the researcher up blind alleys. Ultimately, though, the 
researcher needs to pursue his or her investigation until the questions have 
been answered and things can be explained. A consequence of this is that the 
size and composition of the sample are not completely predictable at the outset. 


Link up with Grounded theory, Chapter 6 
"oc 


Good practice: the cumulative approach and sample size estimates 


The fact that the exact size of the sample cannot be stated with certainty at 
the start of research does not mean that the researcher should start with no 
idea of the final numbers to be included in the sample. Indeed, it makes good 
practical sense for researchers using a cumulative approach to have some 
broad idea of the likely size of their accumulated sample despite their desire to 
pursue threads of enquiry through to a point where they are satisfied they have 
enough information. A shrewd look at the time and resources available, and 
some reading of similar studies, will help to give a reasonable indication 
before the research starts. 


Advantages of surveys and sampling 


There are four main advantages to be gained from the use of surveys and 
sampling in social research. 


1 A focus on empirical data. As an approach to social research, the emphasis 
tends to be on producing data based on real-world observations. The very 
notion of a survey suggests that the research has involved an active attempt 
by the researcher to go out and look and to search. Surveys are associated 
with getting information ‘straight from the horse’s mouth’. 

2 Can collect both quantitative data and qualitative data. The survey approach 
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lends itself to being used with particular methods, such as the self- 
completion questionnaire, which can generate large volumes of quantitative 
data that can be subject to statistical analysis. For this reason, researchers 
who find that quantitative data will suit their needs will find themselves 
drawn to the survey approach. However, surveys can just as easily produce 
qualitative data, particularly when used in conjunction with methods such 
as an interview. As we have seen, the use of non-probability sampling in 
small-scale surveys is often used by researchers to collect qualitative data 
from an exploratory sample or as part of a grounded theory enquiry. 

3 Wide and inclusive coverage. The notion of a survey involves the idea of a 
span of vision which is wide and inclusive. In different ways, this is true for 
both large-scale questionnaire surveys and for small-scale interview surveys. 
In the case of large-scale questionnaire surveys, the breadth of coverage is an 
advantage in getting representative data from which generalizations can be 
made about the wider population. If the coverage is suitably wide and inclu- 
sive, it gives credibility to generalized statements made on the basis of the 
research. In the case of small-scale interview surveys, researchers can use 
sampling techniques which allow them to build up a picture based on data 
that includes the full range of relevant items or people, perhaps selectively 
including the unusual or extreme ‘outliers’ that are seen as likely to provide 
particularly useful information. 

4 Costs and time. Surveys can prove to be an efficient and relatively inexpensive 
means of collecting data. In the case of small-scale surveys, the researcher’s 
time can often be the only significant cost involved and, although large- 
scale surveys can involve substantial costs, compared with strategies such 
as experiments and ethnography, they can produce a mountain of data in 
a short time for a relatively low cost. Results can be obtained over a fairly 
short period of time and the researcher can set a finite time-span for this, 
which is very useful when it comes to planning the research and delivering 
the end-product. The exception to this is where small-scale research uses 
purposive, theoretical or snowballing sampling techniques. The duration 
of the research can be difficult to predict and it will normally take longer to 
complete than other forms of survey. 


Disadvantages of surveys and sampling 


1 Tendency to focus on data more than theory. With its emphasis on collecting 
empirical data there is a danger that the ‘data are left to speak for them- 
selves’ without an adequate account of the implications of those data for 
relevant issues, problems or theories. The significance of the data can 
become neglected. There is nothing inevitable about this, and good survey 
research certainly does devote attention to the analysis of data. With 
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grounded theory approaches, indeed, there is an inbuilt process for theor- 
izing the data. An obsession with the data accumulation and data descrip- 
tion, none the less is a danger that the survey researcher needs to be aware of 
and avoid. 

2 Detail and depth of the data. The data produced through large-scale surveys 
are likely to lack detail or depth on the topic being investigated. This is 
almost inevitable. Such surveys tend to forfeit depth in favour of breadth 
when it comes to the data that are produced, especially where quantitative 
data are used. This is not true, however, of small-scale qualitative surveys 
using interview methods. Where the survey uses interview methods, it can 
produce data that are rich and detailed, although in this case the trade-off is 
that the number of people taking part in the survey will be much smaller. 

3 Easily ignored. Response rates from surveys are often quite low and getting a 
reasonable response rate can be quite a challenge for the researcher. This is 
particularly the case with large-scale surveys where potential participants 
are selected at random and where the researcher has no personal contact 
with the potential participants. It is easy to ignore a researcher’s request for 
help that comes through the post or via email. Where selection to a survey 
involves personal contact, such as when the researcher uses snowball sam- 
pling or forms of purposive and theoretical sampling, the response rates 
tend to be far higher. 
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Checklist for surveys and sampling 


When undertaking surveys and sampling you should feel confident about 
answering ‘yes’ to the following questions: 


Does the type of survey being used suit the particular 
aims of the research? 


Is the response rate to the survey comparable with that 
obtained by other similar surveys? 


Have efforts been made to see if there are significant 
differences between the respondents and the non-respondents? 


Is it clear whether the sample is intended to be 
representative or exploratory? 


Is it clear whether the sample selection is based on 
probability or non-probability sampling? 


Has the specific sampling technique been explicitly identified? 


Does the sample selection technique fit comfortably with 
the purpose for which the sample is being used (representative or 
exploratory)? 


If aiming to produce a representative sample, has the 
sampling frame been evaluated in terms of how far it is: 


e complete and inclusive? 
e up-to-date? 
e relevant for the research topic? 


Is the sample size appropriate in relation to: 


the type of sample (representative or exploratory)? 
the desired level of accuracy? 

the likely non-response rate? 

the subdivisions in the data? 

the available resources for data collection? 


If the sample size is less than 30, have the implications 
of this for statistical analysis been recognized in the research? 


© M. Denscombe, The Good Research Guide. Open University Press. 


CASE STUDIES 


What is the case study approach? e When is it appropriate to use a 
case study approach? e What is a ‘case’? e The relevance of a case 
study e The selection of cases e Can you generalize from a case 


study? e Advantages of the case study approach e Disadvantages of 
the case study approach e Checklist for the case study approach 


Case studies focus on one (or just a few) instances of a particular phenomenon 
with a view to providing an in-depth account of events, relationships, experi- 
ences or processes occurring in that particular instance. The use of case studies 
has become extremely widespread in social research, particularly with small- 
scale research. When researchers opt for a case study approach, they buy into a 
set of related ideas and preferences which, when combined, give the approach 
its distinctive character. True, many of the features associated with the case 
study approach can be found elsewhere and are not necessarily unique to this 
strategy. However, when brought together, they form a broad approach to 
social research, with an underlying rationale for the direction and planning of 
an investigation that separates it from the rationale for survey research or the 
rationale for experimental research. 


What is the case study approach? 


Spotlight on one instance or a few instances 


The starting point and arguably the defining characteristic of the case study 
approach is its focus on just one instance of the thing that is to be investigated. 
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Occasionally, researchers use two or more instances but, in principle, the idea 
of a case study is that a spotlight is focused on individual instances rather than 
a wide spectrum. The case study approach, then, is quite the opposite of any 
mass study. The logic behind concentrating efforts on one case rather than 
many is that there may be insights to be gained from looking at the individual 
case that can have wider implications and, importantly, that would not have 
come to light through the use of a research strategy that tried to cover a large 
number of instances — a survey approach. The aim is to illuminate the general 
by looking at the particular. 


In-depth study 


The prospect of getting some valuable and unique insight depends on being 
able to investigate things in a way that is different from, and in some senses 
better than, what is possible using other approaches. What a case study can do 
that a survey normally cannot is to study things in detail. When a researcher 
takes the strategic decision to devote all his or her efforts to researching just 
one instance, there is obviously far greater opportunity to delve into things in 
more detail and discover things that might not have become apparent through 
more superficial research. 


Focus on relationships and processes 


Relationships and processes within social settings tend to be interconnected 
and interrelated. To understand one thing it is necessary to understand many 
others and, crucially, how the various parts are linked. The case study approach 
works well here because it offers more chance than the survey approach of 
going into sufficient detail to unravel the complexities of a given situation. It 
can deal with the case as a whole, in its entirety, and thus have some chance 
of being able to discover how the many parts affect one another. In this 
respect, case studies tend to be ‘holistic’ rather than deal with ‘isolated factors’. 
It follows from this that in case studies there is a tendency to emphasize 
the detailed workings of the relationships and social processes, rather than 
restrict attention to the outcomes from these. Quite rightly, a good case 
study plays to its strengths. End-products, outcomes and results all remain of 
interest to the case study researcher, but if attention were not given to the 
processes which led to those outcomes, then the value of the case study 
would be lost. The real value of a case study is that it offers the opportunity 
to explain why certain outcomes might happen - more than just find out 
what those outcomes are. For example, when one is looking at the turn- 
over of labour in an organization, the strength of a case study approach 
would be that it could investigate the processes that explain the actual level of 
turnover — the intricate details of the recruitment policy, staff development, 
nature of the work, levels of pay, background of the workers, etc., and how all 
these are interrelated - all this over and above giving a detailed description 
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of what the facts of the situation are with respect to labour turnover (the 
outcome). 


Natural setting 


‘The case’ that forms the basis of the investigation is normally something that 
already exists. It is not a situation that is artificially generated specifically for 
the purposes of the research. It is not like an experiment where the research 
design is dedicated to imposing controls on variables so that the impact of a 
specific ingredient can be measured. As Yin (2009) stresses, the case is a ‘natur- 
ally occurring’ phenomenon. It exists prior to the research project and, it is 
hoped, continues to exist once the research has finished. 


Multiple sources and multiple methods 


One of the strengths of the case study approach is that it allows the researcher 
to use a variety of sources, a variety of types of data and a variety of research 
methods as part of the investigation. It not only allows this, it actually invites 
and encourages the researcher to do so. Observations of events within the case 
study setting can be combined with the collection of documents from official 
meetings and informal interviews with people involved. Questionnaires might 
be used to provide information on a particular point of interest. Whatever is 
appropriate can be used for investigating the relationships and processes that 
are of interest. 


$% 


Key point: the case study strategy 


The decision to use a case study approach is a strategic decision that 
relates to the scale and scope of an investigation, and it does not, at least 
in principle, dictate which method or methods must be used. Indeed, a 
strength of the case study approach is that it allows the use of a variety of 
methods depending on the circumstances and the specific needs of the 
situation. 


Case study research characteristically emphasizes 


Depth of study rather than Breadth of study 
The particular rather than The general 
Relationships/processes rather than Outcomes and 
end-products 
Holistic view rather than Isolated factors 
Natural settings rather than Artificial situations 


Multiple sources rather than One research method 
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When is it appropriate to use the case study approach? 


The case study approach works best when the researcher wants to investigate 
an issue in depth and provide an explanation that can cope with the complex- 
ity and subtlety of real life situations. In particular, it lends itself to the study of 
processes and relationships within a setting. The use of more than one research 
method sits comfortably with the case study approach although, in practice, 
the use of the case study approach has been aligned with qualitative research 
far more than it has with quantitative research. 

Case studies have been used for a wide range of purposes within social 
research. Predominantly, they have been used in relation to the discovery of 
information (following an inductive logic). Less commonly, they have been 
used in relation to the testing of theory (following a deductive logic). It is quite 
possible, in practice, for any particular case study to incorporate elements of 
both. Table 2.1 depicts the possible ways in which case studies might be used 
but does not imply that any particular case study must be restricted to just the 
goals associated with just one category. 


Table 2.1 The uses of a case study 


Discovery led 

Description Describes what is happening in a case study setting (e.g. events, 
processes and relationships) 

Exploration Explores the key issues affecting those in a case study setting (e.g. 
problems or opportunities) 

Comparison Compares settings to learn from the similarities and differences between 
them 

Theory led 

Explanation Explains the causes of events, processes or relationships within a setting 

Illustration Uses a case study as an illustration of how a particular theory applies in a 
real-life setting 

Experiment Uses a case study as a test-bed for experimenting with changes to specific 
factors (or variables) 


What is a ‘case’? 


The case study approach can use a wide range of social phenomena as the unit 
of analysis. It can be based on things like an individual, an organization, an 
industry, a workplace, an educational programme, a policy or a country. The 
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range of potential ‘cases’ is very wide. To qualify as something that lends itself 
to case study research, however, it is crucial that the unit has distinct boundar- 
ies. The use of a case study approach assumes that the researcher is able to 
separate some aspect of social life so that it is distinct from other things of the 
same kind and distinct from its social context. Without some notion of a 
boundary, it becomes impossible to state what the case is. If the case has no 
end-point, no outside, then it bleeds into other social phenomena and ceases 
to have any distinct identity. Fundamentally, 


e a ‘case’ needs to be a fairly self-contained entity; 
e a ‘case’ needs to have fairly distinct boundaries. 


Good practice: defining the case 


Good case study research needs to contain a clear vision of the boundaries to 
the case and provide an explicit account of what they are. 


The relevance of a case study 


The case study approach generally calls for the researcher to make choices 
from among a number of possible events, people, organizations, etc. The 
researcher needs to pick out one example (or just a few) from a wider range of 
examples of the class of thing that is being investigated: the choice of one 
school for a case study from among the thousands that could have been 
chosen; the choice of one sexually transmitted disease clinic from among the 
many in hospitals across the country; the focus on one bus company from 
among the hundreds throughout the nation which provide local public trans- 
port services. Whatever the subject matter, the case study normally depends 
on a conscious and deliberate choice about which case to select from among a 
large number of possibilities. Two points follow from this. 

First, cases are not randomly selected: they are selected on the basis of 
known attributes. As a distinct alternative to the randomization principle 
associated with classic experiments and large-scale surveys, instances selected 
for a case study are chosen on the basis of their distinctive features. Instances 
selected for an experiment or a large-scale survey are chosen on a random basis 
to ensure as far as possible that they do not represent any specific factors relat- 
ing to the variable that is being studied, but quite the opposite is true when it 
comes to case study research. 

Second, the criteria used for the selection of cases need to be made explicit 
and need to be justified as an essential part of the methodology. Initially, 
this involves identifying the key features of the case and providing relevant 
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information about that feature. For example, if the research topic was ‘small 
firms in the automotive industry’ the criterion for the selection of any particu- 
lar organization as the basis for a case study would be ‘size’. If size is defined in 
terms of the number of employees, then details would need to be given about 
the number of employees in the selected firm and how this compares with 
(1) definitions of small firms (the theory) and (2) a profile of the size of other 
firms throughout the automotive industry. Such details are vital because they 
justify the choice of the particular firm as a suitable example of the broader 
category of the thing being studied (small firms) and they form the basis for 
any generalizations that can be made from the case study findings. 


Good practice: choosing the case 


A case study should be chosen deliberately on the basis of specific attributes to 
be found in the case — attributes that are particularly significant in terms of the 
practical problem or theoretical issue that the researcher wants to investigate. 


The selection of cases 


All case studies need to be chosen on the basis of their relevance to the prac- 
tical problems or theoretical issues being researched. But, having established 
this, the selection of the specific case will also reflect a range of other consider- 
ations as well. These other factors concern the way the case study is to be used 
and the amount of flexibility the researcher is able to exercise in the selection 
of the specific case. 


The way the case will be used 
Typical instance 


The most common justification to be offered for the selection of a particular 
case is that it is typical. The logic being invoked here is that the particular case 
is similar in crucial respects with the others that might have been chosen, and 
that the findings from the case study are therefore likely to apply elsewhere. 
Because the case study is like most of the rest, the findings can be generalized 
to the whole class of thing. 


Extreme instance 


A case might be selected on the grounds that, far from being typical, it provides 
something of a contrast with the norm. An illustration of this would be the 
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selection of an organization which is notably smaller or notably larger than 
usual. Among local authorities in the country, a very small one might be 
chosen for a case study, and the logic for doing so would be that this would 
allow the influence of the factor (size) to be more easily seen than it would be 
in the average size authority. In an extreme instance, a specified factor is seen 
in relief — highlighted in its effect. 


Test-site for theory 


The logic for the selection of a particular case can be based on the relevance of 
the case for previous theory. This is a point Yin (2009) stresses. Case studies can 
be used for the purposes of ‘theory-testing’ as well as ‘theory-building’, to 
use Layder’s (1993) distinction. The rationale for choosing a specific case, 
then, can be that it contains crucial elements that are especially significant, 
and that the researcher should be able to predict certain outcomes if the theory 
holds true. 


Least likely instance 


Following the idea of test-sites for theory, a case might be selected to test the 
validity of ‘theory’ by seeing if it occurs in an instance where it might be least 
expected. So, for example, a researcher who wants to test the ‘theory’ that 
school teachers place a high value on their autonomy could deliberately select 
a situation where such autonomy would seem to be least valued: a school 
with team teaching in open plan classrooms. If there is evidence supporting 
the ‘theory’ even under such ‘least likely’ conditions, then the ‘theory’ has 
all the more credibility. 


Practical considerations 


Although researchers should never rely on practical reasons as the principal or 
the sole criterion for selecting a case, it would be naive to ignore the fact that, 
in the real world of research, such factors do have a bearing on the selection 
of cases. 


A matter of convenience 


In the practical world of research, with its limits to time and resources, the 
selection of cases is quite likely to include a consideration of convenience. 
Faced with alternatives which are equally suitable, it is reasonable for the 
researcher to select the one which involves the least travel, the least expense 
and the least difficulty when it comes to gaining access. The crucial point here, 
though, is that convenience should only come into play when deciding 
between equally suitable alternatives. Selection on the basis of ‘the first to 
hand’, ‘the easiest’ or ‘the cheapest’ is not a criterion in its own right which 
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can be used to justify the selection of cases. If used on its own, in fact, it would 
almost certainly be a symptom of poor social research. Used properly, it is 
subordinate to the other criteria. 


Intrinsically interesting 


If a case is intrinsically interesting, then it can prove an attractive proposition. 
The findings are likely to reach a wider audience and the research itself is likely 
to be a more exciting experience. There are even some experts in the field who 
would go as far as to argue that selection on the basis of being intrinsically 
interesting is a sufficient justification in its own right (Stake 1995). This goes 
right to the heart of the debate about whether cases are studied ‘in their own 
right’, as Stake would maintain, or for what they reveal about others of the 
kind, and, in light of its controversial nature, it would be rather foolhardy for 
the newcomer or project researcher to use this as the sole criterion for selecting 
a case. The vast majority of social researchers would not see it as a justification 
for selection in its own right. It might work for journalism, but social research 
by most definitions calls for more than just this (Ragin 1994). It is far wiser, 
therefore, to regard any intrinsic interest of the case as a criterion to be used 
when deciding between instances that in all other crucial respects are equally 
suitable, and as a bonus. 


No real choice 


On some occasions, researchers do not really have a great deal of choice when 
it comes to the selection of suitable cases for inclusion in the investigation. 
The choice is more or less dictated by circumstances beyond their control. This 
happens in the following two circumstances. 


The study is part of commissioned research 


Commissioned research might leave the researcher with little leeway in the 
selection of cases. The funder is quite likely to stipulate that the research must 
be linked to a specified organization or activity, leaving no discretion on the 
matter to the researchers themselves. Under such circumstances, there is no 
real choice in the selection of cases. 


There are unique opportunities 


There are times when events occur which provide the researcher with unique 
opportunities. The events themselves, of course, will not be unique; they will 
be instances of a class of such events. The opportunity to study such events, 
however, may be unique. Situations which could not be planned or created 
present themselves as ‘one-off chances’. At one level, this could take the 
form of social catastrophes, such as war, famine or natural disaster. At a more 
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mundane level, the unique opportunities could reflect the unpredictable or 
rare nature of the events. Strikes, for example, might be the kind of class of 
events that could be illuminated through the depth study of individual cases. 
Researchers, though, will have little choice over which instances they select as 
their cases and will necessarily find themselves homing in on such events as 
and when they occur. As a consequence, there is no real element of choice in 
the selection of the cases. 


Can you generalize from a case study? 


The value of a case study approach is that it has the potential to deal with 
the subtleties and intricacies of complex social situations. This potential 
comes from the strategic decision to restrict the range of the study to just 
one or a few cases. When opting for this approach, however, the social 
researcher is likely to confront scepticism about the findings - scepticism 
which arises from doubts about how far it is reasonable to generalize from the 
findings of one case. The researcher will probably find people asking questions 
such as: 


e How representative is the case? 

e Isn't it possible that the findings, though interesting, are unique to the 
particular circumstances of the case? 

e How can you generalize on the basis of research into one instance? 


These are reasonable questions. They reflect the key issue of generalization in 
social research and need to be addressed by anyone who decides to adopt a 
case study approach. They should not be ignored in the hope that readers of 
the research report will overlook the point. Indeed, it is good practice for any 
researcher who decides to choose a case study approach to pre-empt possible 
criticism by addressing the issue head-on. There should be an explicit defence 
against the allegation that you cannot generalize from case study findings, and 
this can use the following three arguments. 


1 Although each case is in some respects unique, it is also a single example of a 
broader class of things. If, for example, the study is based on a small primary 
school, this is to be treated as an instance of other schools which are small 
and which are in the primary sector. It is one of a type (Hammersley 1992; 
Ragin and Becker 1992; Yin 2009). 

2 The extent to which findings from the case study can be generalized to other 
examples in the class depends on how far the case study example is similar to 
others of its type. To pursue the example of the small primary school, the 
applicability of the findings from the case study to other small primary 
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schools will depend on how far the case study example shares with other 
schools in the class (small size, primary sector) features which are significant 
as far as the operation of such schools are concerned. Its catchment area, the 
ethnic origins of the pupils and the amount of staff turnover might be 
regarded as vital factors. If so, the generalizability of the findings from the 
particular case study school to small primary schools in general will depend 
on the extent to which its profile on these factors is typical of those found 
elsewhere. Equally, the case study researcher might wish to stress the extent 
to which the particular example being investigated is unusual, and thus 
emphasize the limits to how far the findings should be generalized to others 
in the class. 


Good practice: generalizations from case studies 


To draw wider inferences from the study of a particular case the researcher 
needs to: 


e identify significant features on which comparison with others in the class 
can be made; and 

e show how the case compares with others in the class in terms of these 
significant features. 


With the example of the small primary school, this means that the 

researcher must obtain data on the significant features (catchment area, the 
ethnic origins of the pupils and the amount of staff turnover) for primary 
schools in general, and then demonstrate where the case study example fits 
in relation to the overall picture. 
Reports based on the case study include sufficient detail about how the case 
compares with others in the class for the reader to make an informed judgement 
about how far the findings have relevance to other instances. When it comes 
to making generalizations on the basis of case studies, some of the responsi- 
bility falls to the reader. The reader of the findings will use the informa- 
tion to make some assessment of how far the findings have implications 
across the board for all others of the type, or how far they are restricted to 
just the case study example. The reader, though, must be provided with the 
necessary information on which to make an informed judgement on this 
matter. 


Comparing a case with others of its type 


Comparison might call for the inclusion of details on the following factors: 


Physical location geographical area, town, building, room, furni- 
ture, decor 
Historical location developments and changes 
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Social location catchment area, ethnic grouping, social class, 
age, sex and other background information 
about the participants 

Institutional location type of organization, size of organization, official 
policies and procedures 


Advantages of the case study approach 


e The main benefit of using a case study approach is that the focus on one ora 
few instances allows the researcher to deal with the subtleties and intricacies of 
complex social situations. In particular, it enables the researcher to grapple 
with relationships and social processes in a way that is denied to the survey 
approach. The analysis is holistic rather than based on isolated factors. 

e The case study approach allows the use of a variety of research methods. 
More than this, it more or less encourages the use of multiple methods in 
order to capture the complex reality under scrutiny. 


Link up with Mixed methods, Chapter 8 ` 

rh 

e In parallel with the use of multiple methods, the case study approach fosters 
the use of multiple sources of data. This, in turn, facilitates the validation of 
data through triangulation. 

e The case study approach is particularly suitable where the researcher has 
little control over events. Because the approach is concerned with investi- 
gating phenomena as they naturally occur, there is no pressure on the 
researcher to impose controls or to change circumstances. 

e The case study approach can fit in well with the needs of small-scale 
research through concentrating effort on one research site (or just a few sites). 

e Theory-building and theory-testing research can both use the case study 
approach to good effect. 


Disadvantages of the case study approach 


e The point at which the case study approach is most vulnerable to criticism is 
in relation to the credibility of generalizations made from its findings. The 
case study researcher needs to be particularly careful to allay suspicions and 
to demonstrate the extent to which the case is similar to, or contrasts with, 
others of its type. 
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e Unwarranted though it may be, case studies are often perceived as producing 
‘soft’ data. The approach gets accused of lacking the degree of rigour 
expected of social science research. This tends to go along with the view of 
case study research as focusing on processes rather than measurable end- 
products, as relying on qualitative data and interpretive methods rather 
than quantitative data and statistical procedures. Often, case studies are 
regarded as acceptable in terms of providing descriptive accounts of the 
situation but rather ill-suited to analyses or evaluations. None of this is 
necessarily justified, but it is a preconception which the case study 
researcher needs to be aware of, and one which needs to be challenged by 
careful attention to detail and rigour in the use of the approach. 

e On the technical side, the boundaries of the case can prove difficult to define 
in an absolute and clear-cut fashion. This poses difficulties in terms of decid- 
ing which sources of data to incorporate in the case study and which to 
exclude. 

e Negotiating access to case study settings can be a demanding part of the 
research process. Research can flounder if permission is withheld or with- 
drawn. In case studies, access to documents, people and settings can gener- 
ate ethical problems in terms of things like confidentiality. 

e It is hard for case study researchers to achieve their aim of investigating 
situations as they naturally occur without any effect arising from their pres- 
ence. Because case study research tends to involve protracted involvement 
over a period of time, there is the possibility that the presence of the 
research can lead to the observer effect. Those being researched might behave 
differently from normal owing to the knowledge that they are ‘under the 
microscope’ and being observed in some way. 


Link up with The observer effect, p. 69 
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Checklist for the case study approach 


When undertaking research which involves the case study approach you 
should feel confident about answering ‘yes’ to the following questions: 


Is the research based on a ‘naturally occurring’ situation? 


Have the criteria for selection of the case (or cases) been 
described and justified? 


Has the case been identified as a particular instance of a 
type of social phenomenon (e.g. kind of event, type of 
organization)? 


Have the significant features of the case been described 
and have they been compared with those to be found elsewhere 
among the type of thing being studied? 


Is the case a fairly self-contained entity? 


Have the boundaries to the case been described and their 
implications considered? 


Has careful consideration been given to the issue of 
generalizations stemming from research? 


Does the research make suitable use of multiple methods 
and multiple sources of data? 


Does the research give due attention to relationships and 
processes, and provide a ‘holistic’ perspective? 
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EXPERIMENTS 


What is an experiment? e Cause and effect e The use of controls 
e Observation and measurement e Laboratory experiments 
e Field experiments e Use of contro! groups and ‘randomized 


controlled trials’ e When is it appropriate to conduct an experiment? 
e Advantages of experiments e Disadvantages of experiments 
e Checklist for the experimental approach 


What is an experiment? 


An experiment is an empirical investigation under controlled conditions 
designed to examine the properties of, and relationship between, specific fac- 
tors. It is the bedrock of research in the physical sciences and is regarded by 
many social researchers as a model of good practice. The idea of an experiment 
tends to be linked with white-coated scientists working in a laboratory, pos- 
sibly using highly sophisticated equipment to measure things with extreme 
precision. The point of conducting an experiment is to isolate individual fac- 
tors and observe their effect in detail. The purpose is to discover new relation- 
ships or properties associated with the materials being investigated, or to test 
existing theories. 

Now, although this draws on a rather idealized image of the way experi- 
ments are conducted in the physical sciences, it is not entirely irrelevant for 
the social sciences. It largely succeeds in capturing the essence of the notion 
and incorporates the three things that lie at the heart of conducting an 
experiment: 
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e The identification of causal factors. The introduction or exclusion of factors 
to or from the situation enables the researcher to pinpoint which factor 
actually causes the observed outcome to occur. 

e Controls. Experiments involve the manipulation of key variables. The 
researcher needs to identify factors that are significant and then introduce 
them to or exclude them from the situation so that their effect can be 
observed. 

e Empirical observation and measurement. Experiments rely on detailed empir- 
ical observation of changes that occur following the introduction of poten- 
tially relevant factors. They also involve the precise measurement of the 
changes that are observed. 


$ 


Key point: experiments as a research strategy 


The essential features of ‘the experiment’ are concerned with the aims 
of the investigation and its design. They do not say anything about how 
exactly the data are to be collected. This is why experiments are better 
seen as a strategy for research rather than a method. The decision to 
use an experimental approach is a strategic one in which the researcher 
decides to investigate the topic under controlled conditions, paying 
careful attention to the meticulous measurement of what goes on. 


Cause and effect 


Experiments are generally concerned with determining the cause of any 
changes that occur to the thing being studied. It is not normally enough to 
show that two things that occur are linked; that they always occur at the same 
time or they always happen in sequence. Useful though it is to know about 
such a relationship, experiments usually aim to discover which of the factors is 
the cause. This requires a distinction to be made between dependent and 
independent variables. It is vital that the researcher has a clear idea of which is 
which, and sets up the experiment in a way which produces results that show 
the distinction between the two: 


e The independent variable is the one that has the impact on the dependent 
variable. Its size, number, structure, volume or whatever exists autono- 
mously, owing nothing to the other variable. A change in the independent 
variable affects the dependent variable. 

e The dependent variable is the factor that alters as a result of changes to the 
independent variable. It literally ‘depends’ on the independent variable. Any 
change in the dependent variable does not affect the independent variable. 
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Example: tobacco smoking 


Tobacco smoking is related to the incidence of lung cancer. Both are variables 
and both appear to be linked to each other: more smoking, higher incidence of 
lung cancer; less smoking, lower incidence. Smoking increases the likelihood 
of lung cancer. Lung cancer does not increase the likelihood of smoking. In 
this example, there is clearly a dependent variable and an independent vari- 
able. Smoking tobacco is the independent variable, and lung cancer is the 
dependent variable. 


The use of controls 


When conducting an experiment the aim is to show that the dependent factor 
(for example, the incidence of lung cancer in the population) responds to 
changes in the independent factor (prevalence of smoking in the society). To 
do this, the researcher needs to be sure it was definitely the prevalence of 
smoking that was responsible for the observed levels of lung cancer and not 
some other factor (e.g. diet or amount of exercise) that actually caused it. This 
requires the experimenter to control variables like smoking, diet, exercise and 
other factors that could perhaps affect the incidence of lung cancer to ensure 
that, of all of them, it is only the one factor, smoking, that could possibly be 
linked to the level of lung cancer in the population. There are a number of 
ways in which experiments can be designed to achieve this each of which 
involves the use of controls. 


Introduce a new factor 


The most straightforward way to isolate the impact of a variable is to intro- 
duce it while keeping all other relevant factors unchanged. Under these 
circumstances — ‘all things being equal’ — it is possible to pinpoint the impact 
of the new factor and to deduce that any observed changes that occur can 
be attributed to the new factor. Since the introduction of the new factor is 
the only thing that has changed, it alone can be held responsible for any 
subsequent changes that occur. In practice, however, researchers face two par- 
ticular difficulties in this regard. First, it can be extremely difficult to ensure 
that none of the other variables change. Second, variables are often linked so 
that a change in one variable might cause changes across a range of other 
variables. 
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Eliminate the factor from the experiment 


Rather than introduce a new factor to the situation, the researcher might find 
it easier to eliminate one factor. So, for instance, a study of the link between 
hyperactivity and diet in children might see the researcher deliberately exclud- 
ing all artificial colourings from the subjects’ diets and then observing any 
change in behaviour. If other factors remain unaltered, it would be logical to 
deduce that any observed change was due to the absence of this factor. 


Hold the factor constant 


In social science, a large number of the key relevant variables come in the form 
of attributes that cannot be written out of the equation through eliminating 
them from the situation altogether. Things like income, weight and age are 
attributes that cannot be eliminated. However, they can be controlled for by 
‘holding the factor constant’. To prevent the factor from having an unwanted 
intrusion on the outcome of the experiment, the researcher can devise a situ- 
ation in which the subjects are all of the same income or of the same weight. 
Observed outcomes from the experiment, as a result, can be deduced to stem 
from some factor other than income or weight. 


Observation and measurement 


It might be argued that any form of research involves observation and meas- 
urement, but the point about observation and measurement as they are 
associated with experiments is that they are conducted under conditions that 
are artificially created (to a greater or lesser extent). They take place under 
conditions which have been manipulated to allow the greatest rigour and 
precision. That is why experiments take place in laboratories, because these 
are purpose-built contexts for the research. They enhance the possibility of 
precision. There are, though, two points that need to be borne in mind in 
relation to observations that form part of an experiment. 

First, the artificial setting of an experiment can heighten people’s sensitivity 
to being observed. Humans have self-awareness in a sense that makes them very 
different from the materials studied in other forms of research. When humans 
become aware that they are the focus of attention for research, there is the very 
real possibility that they will act differently from normal. They might possibly 
become self-conscious or alter their behaviour to take account of the purposes 
of the research. This is known as the ‘observer effect’ or ‘Hawthorne effect’. 


Link up with the Hawthorne studies, pp. 142-143 A 
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Experimenters can take steps to overcome this. They can, for instance, make 
hidden observations from behind a one-way mirror. Or they can disguise 
the real purpose of the research to avoid getting altered behaviour in rela- 
tion to the factor being investigated (see the account of Milgram’s research 
below). But both solutions to the observer effect raise ethical problems and 
need very careful consideration before being used by the newcomer or project 
researcher. 


The observer effect 


People are likely to alter their behaviour when they become aware that they are 
being observed. Unlike atoms, people can become aware of being studied — 
conscious of it — and then react in a way that is not usual. They can: 


e be embarrassed; 
e disguise their normal practice/be defensive. 


To overcome the effect, researchers are generally advised to: 


e spend time on site, so that the researcher becomes ‘part of the furniture’; 
e have minimal interaction with those being observed. 


Laboratory experiments 


The use of controls, the search for causes and the emphasis on detailed obser- 
vation and measurement are seen in their most extreme form in ‘laboratory 
experiments’. Such experiments are characteristically: 


e of relatively short duration; 

e located on site rather than ‘in the field’; 

e involve close control of variables to isolate causal factors; 
e involve meticulous observation and measurements. 


Although not necessarily typical of experiments conducted by psychologists, 
a series of experiments conducted by Stanley Milgram at Yale University 
between 1960 and 1963 offers an example that illustrates how such laboratory 
experiments might operate in the case of social research (Milgram 1974). The 
aim of the series of experiments was to shed more light on the apparent will- 
ingness of ordinary individuals to inflict cruel pain on others when instructed 
to do so by an authority figure. It formed part of an ongoing debate, prompted 
by Nazi atrocities during the Second World War, about whether those who 
inflicted the torture were psychologically different from other people, or 
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whether there is a normal psychological tendency to obey authority, even to 
the point of following instructions to hurt a fellow human being against 
whom you bear no personal animosity. Milgram’s series of experiments sought 
to investigate this under strictly controlled conditions: 


The point of the experiment is to see how far a person will proceed in a 
concrete and measurable situation in which he is ordered to inflict increas- 
ing pain on a protesting victim. At what point will the subject refuse to 
obey the experimenter? 

(Milgram 1974: 3-4) 


An advertisement was put in a local newspaper inviting people aged 20 to 50 
to participate in ‘a scientific study of memory and learning’. The advertise- 
ment offered $4 plus 50 cents taxi fare for those willing to give up one hour of 
their time. Those who took up the offer were met at the university department 
by a person who said he was in charge of the experiment — a smart-looking 
young man in a laboratory coat. The ‘subject’ was then introduced to a man 
who, he was led to believe, was another participant in the experiment. This 
person was in his late fifties, very mild-mannered and inoffensive, and slightly 
overweight: not the kind of person you would want to harm. The whole 
thing, however, was a ‘set-up’. The ‘experimenter’ and the other subject were 
colleagues of Milgram, and they were actually taking part in a carefully 
scripted sequence of events. Their words and actions were part of the experi- 
ment and in accord with a strict plan which was followed with each new 
volunteer who came in off the street. This meant that the only real variable 
was the action of each subject who had volunteered to help. As far as was 
possible, all other aspects of what happened were controlled to be the same on 
each occasion. 

The subjects were told that they were to take part in a study of the effects of 
punishment on learning. One of the two participants was to take the role of 
the ‘teacher’ administering the punishment, the other was to be the ‘learner’. 
It was rigged so that when drawing a piece of paper out of a hat to determine 
which subject took which role, Milgram’s colleague always got the role of the 
learner, ensuring that the real subject was left to undertake the teacher role. At 
this point the ‘learner’ was taken to a room and sat in a chair. He was strapped 
down and electrodes were attached to his arm. The whole thing was delib- 
erately set up to resemble the ‘electric chair’. In an adjoining room the 
unsuspecting subject and the ‘experimenter’ in his lab coat went to an ‘impres- 
sive shock generator’. The subject was given a 45-volt shock from this gener- 
ator to make the whole performance that followed convincing. In fact, it was 
the only real electric shock administered throughout the whole experiment - 
but the subject was not to know this. The learning experiment consisted of the 
‘teacher’ (the subject) reading out a series of paired words (e.g. fat-neck, blue- 
girl, nice-day). A stimulus word was then read out, followed by five others, 
among which was the actual paired word read out in the original series. The 
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learner had to identify which was the correct word out of the five. Each time 
he made a mistake, the subject had to deliver an electric shock by way of 
punishment — purportedly to see if this would encourage better learning. Each 
time a mistake was made, an electric shock 15 volts higher than the one before 
was apparently administered. 

The learner put on a performance. Although he actually received no electric 
shock at all, he acted as though he did. As the voltage of the shocks was 
increased he began to moan and protest — all, of course, to a precise script 
that went with each increase. At 150 volts Milgram’s colleague would cry 
out, ‘Experimenter, get me out of here! I won’t be in the experiment any 
more! I refuse to go on!’ At 180 volts he would shout, ‘I can’t stand the 
pain!’ and by 270 volts he would let out an agonized scream. After 330 volts 
the learner fell silent. The lab-coated experimenter continued to coax the sub- 
ject to give the electric shocks, and each time the ‘learner’ protested he gave a 
carefully scripted reassurance to the real subject: ‘No permanent tissue damage 
is caused; he has to go on because it is part of the experiment.’ 

The experimental conditions were altered through the series of experiments. 
The proximity of the subject to the learner was altered, for example, and the 
nature of the complaints that could be heard by the subject emanating from 
the pained learner, but the basic design continued. Milgram found that ordin- 
ary volunteers were remarkably willing to give dangerously high voltages to a 
person they believed to be another ‘off-the-street’ volunteer, a person who 
was apparently vulnerable to heart attack, an inoffensive person, a person who 
was quite evidently suffering extreme pain and demanding to be released. 
Depending on the specifics of the experiment, between one-half and two- 
thirds of subjects went on to administer up to 450 volts! They did not do so 
with glee; they often protested and expressed disquiet about the whole thing. 
But this proportion of ordinary people could be persuaded by the authority of 
the scientific experimenter in his lab coat to deliver what by any standards 
is an extremely dangerous and painful ‘punishment’ to a vulnerable fellow 
being. 

The psychological explanation for this level of obedience to authority is 
discussed in detail by Milgram, but this is not the point of describing the 
experiment here. The point of describing it is to illustrate the way in which the 
use of an experiment relies on controlling as many of the variables as possible 
to allow an uncontaminated measurement of the factor that is of prime inter- 
est and involves precise measurements. It is very unlikely that these could 
have been achieved had the research not taken place in the laboratory, on 
home territory where the researchers are able to set things up in a very meticu- 
lous fashion and repeat the experiment time and again without interference 
from outside factors. 

Admittedly, most laboratory experiments in the social sciences do not 
achieve the fame and notoriety of the Milgram experiments, for obvious 
reasons. They tend to be less provocative in their topic and design. The 
principles, none the less, remain the same. 
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A/N Caution 


In the case of social research, there are certain limitations to the 
extent to which circumstances can be manipulated. Basically, a 
researcher can do things to chemicals and plants that cannot be con- 
templated with fellow human beings. For the purposes of doing an 
experiment it may be neither feasible nor, indeed, ethical to manipu- 
late people’s lives in order to provide the controlled conditions neces- 
sary for a ‘social science experiment’. People have feelings, people 
have rights. 


Link up with Research ethics: a practical guide, Appendix 1 ` 


Field experiments 


The use of experiments by social researchers would appear to be restricted to 
those situations where they are able to manipulate the situation and impose 
controls on crucial variables. Laboratories clearly are designed to aid this. 
There is, though, something of a dilemma here: 


It is often argued that laboratory experiments are of little use because the 
settings in which they take place are artificial and so the results of the 
ensuing findings have little validity beyond the confines of the laboratory. 

(Bryman 1989: 90) 


It is not just that the data are generated under artificial conditions: they are 
actually shaped to some degree by those artificial conditions. Yet, if researchers 
move outside the laboratories in order to gather data in more natural settings, 
they are, of course, likely to pay a high price in terms of their ability to control 
the variables. Beyond the laboratory, the social researcher has far less prospect 
of managing to manipulate things for the express purpose of a social experi- 
ment. Sociologists cannot control levels of incomes in order to conduct 
experiments on things like poverty. Nor can health educators manipulate 
levels of smoking among adolescent girls in order to study it. Economists can- 
not generate a recession in order to investigate its consequences. It is simply 
not possible to manipulate circumstances like these. It is not feasible, nor 
would it be ethical. 

There are, however, approaches within social research which recognize that 
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it is neither feasible nor ethical to attempt to impose controls on all the rele- 
vant factors, yet which strive to retain the logic of the experimental approach, 
albeit in a watered-down version, beyond the bounds of the laboratory. 
Because they cannot always control all the variables, some social scientists 
have resorted to a ‘quasi-experimental’ approach as a realistic possibility. The 
quasi (‘as if’) experimental approach is conducted in the spirit of the classic 
laboratory experiment, but recognizes that the researcher cannot dictate cir- 
cumstances and needs to take the role of observing events ‘as they naturally 
occur’. Much like weather forecasters, these social scientists tend to rely on 
observing events which occur in circumstances over which they have little or 
no control. Researchers in this case are on the look-out for ‘naturally occur- 
ring’ experiments — situations in which they can see the possibility of observ- 
ing and measuring the impact of isolated variables through circumstances as 
they happen, without imposing artificial controls. 

Naturally occurring experiments are part of the logic of anthropological 
studies of simple, ‘primitive’ societies. As Margaret Mead made the point: 


The study of human development in a primitive society has two advan- 
tages: contrast with our own social environment which brings out differ- 
ent aspects of human nature and often demonstrates that behaviour 
which occurs almost invariably in individuals within our own society is 
nevertheless due not to original nature but to social environment; and a 
homogeneous and simple social background, easily mastered, against 
which the development of the individual may be studied . . . The anthro- 
pologist submits the findings of the psychologist who works within our 
society to the test of observation within other societies. 

([1930] 1963: 212) 


Another example of quasi-experimental research comes in the form of 
ex post facto experiments. In this instance, the starting point for the investiga- 
tion is the ‘effect’, and the researcher sets out to deduce what factors could be 
held to account for this outcome. The idea is to work backwards from the effect 
to find the cause. For example, the researcher might ask, ‘Does imprisonment 
affect earnings after release from detention?’ The research design here would 
be to identify a number of people who had been to prison. This independent 
variable — imprisonment — is not something the research can or should 
manipulate. It is a fait accompli, existing prior to and quite separate from the 
research. However, by comparing ex-prisoners with people who in most other 
respects match them, it is possible to deduce whether or not that factor has 
had a direct effect on the dependent variable: earnings. Or, to give another 
illustration, the starting point for the research might be the discovery of lung 
cancer in particular people. Ex post facto research then sets about trying to 
identify variables that might explain the incidence of lung cancer in these 
people. It might be discovered that, compared with the population as a whole, 
a higher proportion of those with lung cancer smoke tobacco. Controlling for 
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other factors to discount them as possible causes, the researcher might eventu- 
ally deduce that smoking tobacco is the cause of the higher incidence of lung 
cancer. 


Table 3.1 Laboratory and field experiments compared 


Laboratory experiments Field experiments 
Located ‘on site’ Located ‘in the field’ 
Close control of variables Use available possibilities 
Meticulous measurements Structured observations 
Shorter duration Longer duration 


Use of control groups and ‘randomized controlled trials’ 


First, it is necessary to identify two groups of people for the experiment. The 
two groups should be similar in terms of their composition. One group is 
identified as the experimental or ‘treatment’ group, the other is treated as the 
‘control’ group. 

In the case of randomized controlled trials (RCTs), the rule is that there is 
random allocation to the groups. If the research subjects are chosen on a ran- 
dom basis, there should be a tendency for those factors which are not crucial to 
cancel themselves out. The principle here is that by the choice of a large 
enough group of subjects on a random basis, any interference with the results 
through ‘confounding’ factors would cancel themselves out in terms of the 
results for the whole research group. RCTs are regarded as the gold standard 
for research design in the realms of medical and health research concerned 
with clinical trials of drugs and other health interventions. Because the 
research is generally large-scale and involves large numbers in both the 
experimental group and the control group, the random allocation works as a 
way of ensuring that there is no difference between the groups. 


oe 
Link up with Random sampling, p. 27 e 


With smaller-scale social research, selection to either group is more likely to 
be based on known factors where there is a deliberate attempt to ensure that 
the groups are balanced with respect to certain key criteria. Researchers might, 
for example, wish to ensure that the groups are similar with respect to age, sex 
and ethnicity and they could take steps to allocate people to the groups accord- 
ingly. Selection to the groups in this instance is based on purposive sampling. 
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The starting point for the experiment, in either case, is two matched groups 
or samples. The experiment involves introducing a factor to the experi- 
mental group and leaving the other group with no artificially induced 
changes. Having added the new factor, the experimenter can then look at 
the two groups again with the belief that any difference between the groups 
can be attributed to the factor which was artificially induced. Note here that 
it is not the change in the experiment group as such which is important. 
Certain changes over time are likely to happen, irrespective of whether an 
experiment had taken place. Time moves on for all of us. But, if the two 
groups were as identical as possible at the outset, any change of this kind 
in one group will also occur in the other group. So, instead of measuring 
the change from time 1 to time 2, we measure the difference between 
the control group and the experimental group at the end of the experi- 
ment at time 2. Any differences we observe can be logically deduced to 
come from the factor which was artificially induced in the experiment (see 
Figure 3.1). 


FIGURE 3.1 Experiments and the use of control groups 


Pre-test Introduction of Post-test 

(time 1) new factor (E) (time 2) 
Experimental group A Cc 
Control group B —— D 


Time = 


Experimental group and control group are matched groups or samples. 
Factor E causes a change in the experimental group of (C — A) - (D - B). 


When is it appropriate to conduct an experiment? 


The principles of experimentation are ones that can be applied to social 
research in a number of ways. Experiments do not need to be conducted in 
laboratories; there are ways of conducting experiments ‘in the field’ and the 
underlying principles can also be applied to naturally occurring social phe- 
nomena. The comparison of social groups (Mead 1943), of countries (Ragin 
1994), and of case studies (Yin 2009) can all include the principles of 
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experimentation. The crucial factor governing whether or not it is appropriate 
to use experimentation as the basis for the research is whether the circum- 
stances allow researchers to observe the effect of one variable by controlling 
for the effect of others. 

The use of an experimental approach, as we have seen, also depends on: 


e the ability to avoid the contamination of results through the ‘observer 
effect’; 
e the ability to collect findings in an ethical manner. 


Advantages of experiments 


e Repeatable. In the case of laboratory experiments, there is every chance that 
the research will be repeatable. The procedures should have been carefully 
recorded and the variables controlled for. It is in line with the spirit of 
scientific research, lending itself to being checked by being repeated by 
other researchers using identical procedures. 

e Precision. Again, in the case of laboratory experiments, the context for the 
research permits the highest possible level of precision when it comes to the 
measurements that form the basis of the data. 

e Convenience. In the laboratory, the researcher is on ‘home territory’. This has 
advantages in terms of setting up the research, and there are also some 
positive benefits when it comes to costs. The researcher does not need to go 
out into the field and incur the travel costs and the loss of time spent going 
to research sites. 

e Credibility. The use of experiments is regarded by many people, including 
some social scientists, as the most scientific and, therefore, the most cred- 
ible approach to research. It has a status that trades off an association with 
‘hard’ science. 


Disadvantages of experiments 


e Deception and ethics. There are a number of ethical considerations to be 
borne in mind when one is considering the use of experimental research. 
For instance, if the research design employs the use of control groups, 
which is likely, will there be any advantage or disadvantage experienced 
by those in the respective groups? Is it possible that adverse comparisons 
might result through different treatments of experimental and control 
groups? If there is a significant difference, this could pose both ethical and 
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practical/political problems for the researcher. Equally, the experiment’s 
success might depend on keeping its purpose secret from the research 
subjects. This raises a question about the ethics of deception when used in 
social research. 

e Artificial settings. With laboratory experiments there are question marks 
about whether the experimental situation creates conditions comparable 
with the ‘real-world’ situations in which the behaviour/decisions would be 
made, or whether it encourages artificial responses in line with the artificial 
setting. 

e Representativeness of the research subjects. For most purposes, experimental 
researchers will want to use a control group. To use a control group in a valid 
fashion, the researcher needs to be sure that the experimental group and the 
control group constitute a ‘matched pair’. They need to be very closely 
matched in terms of those features which are relevant to the experiment 
and to the broader population from which the research subjects are drawn. 
This can be difficult to achieve. Are the research subjects typical of the 
‘population’ being studied? Do the subjects share the same prior experi- 
ences? Has there been an element of self-selection? 

e Control of the relevant variables. Being able to control the relevant variables 
lies at the heart of the experimental method. However, this places a burden 
on the would-be experimenter to be sure that he or she can do so. And, even 
if it is practically possible, is it ethical? 
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Checklist for the experimental approach 


When undertaking research which involves experiments you should feel 
confident about answering ‘yes’ to the following questions: 


Have the key variables been clearly identified? 


Can the relevant variables be controlled for the purposes 
of designing the experiment? 


Has a clear distinction been drawn between dependent 
and independent variables? 


Does the experimental situation create conditions 
comparable to the ‘real-world’ situations in which the 
behaviour/decisions would be made? 


If a control group has been used, is it closely matched in 
all key respects with the experimental group? 


If the research subjects are fully aware of the nature of 
the experiment in which they are taking part, has ‘the 
observer’ effect been taken into account? 


If the research subjects are not fully aware of the nature 
of the experiment in which they are participating, will they 
be fully informed after the experiment has taken place? 


Is the research ethical in terms of its treatment of those 
involved in the experiment? 


© M. Denscombe, The Good Research Guide. Open University Press. 


ETHNOGRAPHY 


What is ethnography? e The focus of ethnography e Ethnography: 
description and theory e Reflexivity: ethnographers as part of the world 
they seek to describe e Putting the researcher’s self into ethnographic 


research e Access to fieldwork settings e Advantages of ethnography 
e Disadvantages of ethnography e Checklist for enthnographic research 


What is ethnography? 


The term ethnography literally means a description of peoples or cultures. It has 
its origins as a research strategy in the works of the early social anthropologists, 
whose aim was to provide a detailed and permanent account of the cultures 
and lives of small, isolated tribes. Such tribes were seen, with some justifica- 
tion, as ‘endangered species’, and the social anthropologists saw the need to 
map out those cultures before they became contaminated by contact with the 
industrial world or withered away to extinction. 

The image of the pith-helmeted outsider dressed in khaki shorts arriving on 
the shores of some remote and exotic palm tree island to set up camp and study 
the lives of the ‘native’ has become legendary — largely through the works of 
people like Bronislaw Malinowski (1922) and Margaret Mead (1943). The con- 
cerns of such social anthropologists and the research strategy they employed 
set the scene for much of what is undertaken as ‘ethnography’ today. 

Ethnography, based on the early anthropological origins of the term and on 
subsequent developments by influential classics in the field (e.g. Whyte [1943] 
1981), has the following characteristics: 
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e It requires the researcher to spend considerable time in the field among the 
people whose lives and culture are being studied. The ethnographer needs 
to share in the lives rather than observe from a position of detachment. 
Extended fieldwork allows for a journey of discovery in which the explan- 
ations for what is being witnessed emerge over a period of time. 

e Routine and normal aspects of everyday life are regarded as worthy of con- 
sideration as research data. The mundane and the ordinary parts of social 
life are just as valid as the special events and ceremonies which can all too 
easily capture our attention. 

e There is special attention given to the way the people being studied see their 
world. Quite distinct from the researcher’s analysis of the situation, the eth- 
nographer is generally concerned to find out how the members of the group/ 
culture being studied understand things, the meanings they attach to happen- 
ings, the way they perceive their reality. ‘To grasp the native’s point of view, 
his relation to life, to realize his vision of his world’ (Malinowski 1922: 25). 

e There is an emphasis on the need to look at the interlinkages between the 
various features of the culture and to avoid isolating facets of the culture 
from the wider context within which it exists. Ethnography generally pre- 
fers a holistic approach which stresses processes, relationships, connections 
and interdependency among the component parts. 

e There is some acknowledgement that the ethnographer’s final account of 
the culture or group being studied is more than just a description — it is a 
construction. It is not a direct ‘reproduction’, a literal photograph of the 
situation. It is, rather, a crafted construction which employs particular 
writing skills (rhetoric) and which inevitably owes something to the eth- 
nographer’s own experiences. 


$ 


Key point: the holistic perspective of ethnography 


‘One of the first conditions of acceptable Ethnographic work certainly is 
that it should deal with the totality of all social, cultural and psychological 
aspects of the community, for they are so interwoven that not one can be 
understood without taking into consideration all the others.’ 

(Malinowski 1922: xvi) 


The focus of ethnography 


Alien lifestyles: the insider’s point of view 


As a topic, ethnography refers to the study of cultures and groups — their 
lifestyle, understandings and beliefs. In doing so, ethnography tends to 
emphasize the importance of understanding things from the point of view of those 
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involved. Rather than explaining things from the outsider’s point of view, there 
is a concern to see things as those involved see things — ‘to grasp the native’s 
point of view’. Early anthropologists used to study relatively small groups in 
societies which were different from their own. They hoped the comparison 
would illuminate aspects of their own society as well as explain the ‘primitive’ 
society where their fieldwork was based. Ethnography, in this respect, 
resembles an anthropological approach, but it does not restrict its attention to 
remote tribes in distant lands. Indeed, the most popular development of eth- 
nography in recent times has been its application to lifestyles, understandings 
and beliefs within ‘our own’ society. The element of comparison and contrast, 
though, is retained as an underlying facet of ethnographic research. 


Link up with Field experiments, p. 72 
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The exotic and the routine 


In its early days, ethnography ‘within our own society’ tended to focus on 
groups who were relatively small in number, and tended to be somewhat alien 
to the mainstream of society. The anthropological stance was applied to devi- 
ant subgroups, oddball cultures that stood out as different from the norm. 
And, just as with the study of ‘natives’ in far-off lands, there was an immediate 
attraction for studying such groups. They offered something intrinsically 
interesting in the way their lifestyles seemed quaint, crazy, even exotic com- 
pared with the everyday experience of those who studied the groups and those 
who read the resulting books. The title of one of the classics illustrates the 
point. Margaret Mead’s (1943) study, originally published in 1928, was titled 
Coming of Age in Samoa: A Study of Adolescence and Sex in Primitive Societies. Such 
a style of research, in a sense, pandered to a certain voyeurism. 

When applied to ‘our own’ society, the focus was on ‘deviant’ groups, such 
as hobos, alcoholics, drug users, religious sects, street gangs, and the like. There 
remains, it is true, legacies of an interest in the exotic and the special, with 
ceremonies and the unusual features of social life. A recent ethnography, for 
example, focused on Wall Street investment bankers exploring their everyday 
experiences, beliefs and the culture surrounding their lives as financial high- 
flyers (Ho 2009). However, the routine and the mundane, the normal and the 
unspectacular facets of social life have become recognized as equally valid 
topics for ethnographic enquiry. In recent times, attention has been refocused 
on to more routine, mainstream aspects of social life — for example, life in 
classrooms (Woods 1979) or life on a building site (Reimer 1979). 


Life on the Internet 


The Internet is more than a physical network and electronic technology. It 
enables a cyberworld populated by particular kinds of people. And this has 
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attracted the attention of some ethnographers. For them, the Internet gives 
rise to a social phenomenon that needs to be investigated and explained. In 
this vein, research has emerged on: 


e the cultural, social and technological conditions giving rise to the Internet 
and its widespread use; 

e the personal attributes of those who use the Internet and the particular 
facilities it provides. 


For others, the Internet is of particular interest because it is seen as a whole 
new aspect of social existence inhabited by communities with their own dis- 
tinctive cultures, styles of interaction and modes of personal communication. 
The interest in ‘online cultures’ and ‘online communities’ has given rise to 
what Hine (2000) has called ‘virtual ethnography’ and Kozinets (2010) has 
termed ‘netnography’. Ethnography, in such cases, focuses on the culture of 
cyberspace. 


Life history 


A life history approach involves the in-depth study of individuals, social groups 
or communities. Mainly, it has come to be associated with the study of indi- 
viduals. The aim of the approach is to portray the lives of specific people, to map 
how their experiences change over time, and to link these experiences with the 
cultural, social or historical context in which they occur. The approach came to 
prominence through the work of sociologists at the University of Chicago dur- 
ing the 1920s to 1940s who paved the way by conducting detailed empirical 
studies of individuals, groups and communities (Denzin 1970; Bulmer 1984). 
These ethnographies traced the changes and developments that took place over 
a period of time and paid particular attention to experiences of those whose 
lives were being studied. More recently the approach has been associated with 
feminist research (Devault 1990) and studies of individuals in relation to 
occupations (Woods 1993) and health (Haglund 2004). 

Like biographies, life histories can be based on the whole of a person’s 
lifespan from beginning to end, but can also be used to cover portions of lives — 
extended and substantial parts of people’s lives but not the whole of the life. 

There are two themes running through most life history research, both of 
which fit closely with ethnography. The first is that the research provides a 
‘thick’ description of the life of the person. It includes the kind of fine detail 
that allows the reader to see the complex and multi-layered facets of the per- 
son’s life. The second is that the individual’s understanding of things is treated 
as a crucial part of the life history. As Marshall and Rossman (1999: 121) put it, 
the life history approach ‘understands a culture through the history of one 
person’s development of life within it, told in ways that capture the person’s 
own feelings, views, and perspectives’. 

It follows from this that life histories can be phenomenological in their 
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approach, treating the core of the research as an attempt to provide an 
insider’s view of life in relation to a particular culture or a particular time in 
history. The researcher’s question is ‘How does someone living at that time 
in that culture really understand things?’ and his/her task is to use the person’s 
reflections on their life as the basis for producing an account that enables the 
reader to see things through the eyes of the person. 

Contrasting slightly with this, some life histories are used descriptively, their 
main aim being to use the personal biography as a means to reveal new things 
about the cultural, historical and social circumstances within which the per- 
son lives (or lived). The research, in this instance, attempts to provide insights 
into the lifestyle and beliefs of a particular culture or group. The idea is that we 
can learn what things were like from someone who had first-hand experience 
of being there or who had lived during that period. 

Other researchers approach life histories in a more analytic fashion. They are 
more inclined to treat the life history as something that should be interpreted 
in terms of the wider social structure within which it occurs. There is an issue, 
here, about whether the life history entails recounting the story of a person’s 
life in the way that they see it, or whether there is a need to bring alternative 
sources to bear by way of corroborating or challenging the person’s version of 
their own life. Denzin argues that: 


The life history must contain reports from persons other than the subject. 
Not only must users of life histories triangulate methodologies, but they 
must also triangulate perspectives: ... Unless the perspectives of others 
are brought to bear upon the subject’s statements, observers have no way 
of assessing the idiosyncrasies of those interpretations. 

(1970: 238) 


Despite these differences in emphasis, what life histories share is an approach 
that uses the in-depth study of an individual’s life in order to get a grasp on 
their experiences and thinking in a way that links these to the cultural, social 
and historical circumstances in which they occur. 

Life histories generally make use of in-depth interviews to collect the data. 
They can also make use of secondary source data such as public archive 
material, private archive material, diaries, letters and other forms of docu- 
mentary data to build the life history. As with most ethnographic research, a 
distinctive feature of the data collection is that it takes place over a relatively 
long period and a life history is not something that can be completed quickly. 


Usually, it will be gathered over a number of years with gentle guidance 
from the social scientist, the subject either writing down episodes of life or 
tape recording them. At its best, it will be backed up with intensive obser- 
vation of the subject’s life, interviews with friends and perusals of letters 
and photographs. 

(Plummer 2001: 14) 


84 THE GOOD RESEARCH GUIDE 


Perhaps the most interesting question of methods in relation to the life 
history approach is not how the data are collected, but how the informants are 
chosen. As Plummer (2001: 133) puts it so well: ‘Who from the teeming mil- 
lions of the world population is to be selected for such intensive study and 
social science immortality?’ Obviously, it will be people who are ‘information 
rich’ — people with relevant experiences, fascinating insights and stories to tell. 
The way the researcher finds such people, people who are willing to open up 
their lives to the inquisitive gaze of the researcher, can follow a process of 
‘selective sampling’. The researcher will be looking in certain appropriate 
places and will have in mind some kind of person who might have the kind of 
life history that will be revealing and informative. However, when it comes to 
finding the ‘tight’ person — the one who turns out to have the treasure trove of 
life experiences — famous instances of life history research show that luck has a 
large part to play. 


Link up with Narrative analysis, p. 293 A 


Ethnography: description and theory 


At first glance it would seem easy to associate ethnography with the task of 
providing straightforward descriptions of things witnessed first-hand in the 
field by the researcher. The crucial factors, from this perspective, are the depth 
and detail of the description, the accuracy of what it portrays and the insights 
it offers to readers about the situation being studied. The purpose of the 
ethnographic research is to produce detailed pictures of events or cultures — 
descriptions which stand in their own right without the need to worry about 
how representative the situation is or what the broader implications might be 
in terms of other events or cultures of the type, or of contributing to wider 
theories. The ethnography, from this stance, is a stand-alone ‘one-off’ that is to 
be judged by the depth of its portrayal and the intricacy of its description. This 
is an idiographic approach to ethnography. 


Link up with Phenomenology, Chapter 5 - 
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A challenge to the idiographic stance within ethnography comes from those 
who question the value of producing numerous stand-alone descriptions if 
there is no attempt to derive something from them which goes beyond the 
specifics of the situation and which can, in some way or other, link to broader 
issues. If each ethnographic study produces a one-off, isolated piece of infor- 
mation, these pieces cannot contribute to the building up of any generalized 
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knowledge about human societies, it is alleged. They take the position that 
ethnographic research should be nomothetic — undertaken quite deliberately to 
develop more generalized and ‘theoretical’ conclusions which will apply out- 
side the confines of the individual ethnographic study. Some researchers 
would go still further. They would argue that the purpose of ethnographic 
research should be to shed some light on an area of life whose significance 
depends on a theory; to elaborate on a theory or even check on whether the 
theory really does hold true and explain things as they happen in ‘real life’ 
(Porter 1993). 

So, at one end of the spectrum there are those who regard the main purpose 
of ethnography as providing rich and detailed descriptions of real-life situ- 
ations as they really are. At the other end of the spectrum there are those who 
see the role of ethnographic fieldwork as a test-bed for theories — a means of 
developing theories by checking them out in small-scale scenarios. And 
somewhere towards the middle of the spectrum lies the pragmatic view ‘that 
“idiographic” and “nomothetic” approaches are not mutually exclusive, and 
that we can have both rich and intensive description and generalizability’ 
(Woods 1979: 268). Advocates of the middle position are keen to hold on to 
the idiographic aspect of ethnographic research in as much as it provides a 
valuable and distinct kind of data — the detailed descriptions of specifics based 
on first-hand observation in naturally occurring situations. But they also rec- 
ognize the need for conclusions to be drawn from the research which have 
implications beyond the bounds of the particular events, customs or people 
studied. In the words of Hammersley (1990: 598), the ‘descriptions must 
remain close to the concrete reality of particular events but at the same time 
reveal general features of human social life’. 


Good practice: generalizing from ethnographies 


To accommodate the need for ethnographies to combine detailed description 
of particular settings with the need to produce findings that have some general 
implications beyond the specifics of the situation studied, ethnographies 
should: 


e compare their findings with those of other similar ethnographies; 

e consider how their findings tie in with, or contradict, existing relevant theor- 
ies and understanding about human social behaviour; 

e explain the significance of the topic in relation to the beliefs and priorities 
of the researcher’s own culture. 
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Reflexivity: ethnographers as part of the world 
they seek to describe 


Ethnographers tend to be very sensitive to the matter of reflexivity and the 
way it affects their perception of the culture or events they wish to describe. 
What concerns them is that the conceptual tools they use to understand the 
cultures or events being studied are not, and can never be, neutral and passive 
instruments of discovery. 

As researchers, the meanings we attach to things that happen and the lan- 
guage we use to describe them are the product of our own culture, social back- 
ground and personal experiences. Making sense of what is observed during 
fieldwork observation is a process that relies on what the researcher already 
knows and already believes, and it is not a voyage of discovery which starts 
with a clean sheet. We can only make sense of the world in a way that we have 
learnt to do using conceptual tools which are based on our own culture and 
our own experiences. We have no way of standing outside these to reach some 
objective and neutral vantage point from which to view things ‘as they really 
are’. To an extent, we can describe them only ‘as we see them’, and this is 
shaped by our culture, not theirs. 

The question that taxes ethnographers is this: ‘How far can my description 
of the culture or event depict things from the point of view of those involved 
when I can only use my own way of seeing things, my own conceptual tools, 
to make sense of what is happening?’ 

Broadly, ethnographers are conscious of the way in which their account of 
any particular lifestyle or beliefs is a construction based upon the researcher’s 
interpretation of events. Even when it comes to writing up the ethnography, 
there are deep-rooted doubts about claims to objectivity. The outcome of 
ethnographic research is generally recognized by those who write it as being a 
creative work in its own right — the outcome of interpretation, editing and 
skilful writing techniques as well as a reflection of the reality of the situation 
they set out to study (Woods 2006). Inescapably, the account is partial - an 
edited and abridged version — which owes something to the ethnographer as 
well as the culture or events observed. There is nothing new to this. 
Malinowski appreciated the point many years ago. 


In ethnography, the distance is often enormous between the brute 
material of information — as it is presented to the student in his own 
observations, in native statement, in the kaleidoscope of tribal life - and 
the final authoritative presentations of the results. The ethnographer has 
to traverse this distance in the laborious years between the moment when 
he sets foot upon a native beach, and makes his first attempts to get 
into touch with the natives, and the time when he writes down the final 
version of his results. 

(Malinowski 1922: 4) 
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Putting the researcher’s ‘self’ into ethnographic research 


One of the characteristic features of ethnography is the significance it attaches 
to the role of the researcher’s ‘self’ in the process of research. The researcher’s 
identity, values and beliefs become part of the equation - a built-in compon- 
ent that cannot be eliminated as an influence on the end-product findings of 
the project. 

Ethnographic research, therefore, calls for a certain degree of introspection on 
the part of the researcher. He or she needs to reflect upon the way that back- 
ground factors associated with personal experiences, personal beliefs and social 
values may have shaped the way that events and cultures were interpreted. 

However, the ethnographic researcher needs to go beyond mere reflection. 
This is, in a sense, a private activity. Necessary though it is, it needs to be 
portrayed more publicly if it is to support the research outcomes. Researchers 
need to supply their readers with some insights into the possible influence of 
the researcher’s self on the interpretation of events or cultures. There needs to 
be a public account of the self which explores the role of the researcher’s self. 
This account of the self will vary in the amount of detail it provides, depending 
on the nature of the research topic, the kind of methods used and the audience 
for whom the ethnography is written. It is obviously a matter of judgement 
here about the extent to which the account should delve to the depths of 
detail about researchers’ personal matters, and the amount of detail that the 
readership needs to know in order to arrive at a reasonable evaluation of the 
likely impact of the ‘self’ on the research. 


Good practice: reflecting on the researcher’s ‘self’ 


Ethnographies require some reflection by the researcher on how his/her biog- 

raphy, values and experiences might have a bearing on the nature of the study. 

This ‘public account of the role of the self’ can be included in the methodology 

section and possibly in the Preface or Introduction to reports of the findings. 
As a broad guideline, the following factors might be considered: 


e personal beliefs relating to the topic (politics, values, standpoint); 

e personal interests in the area of investigation (vested interest, history of 
events); 

e personal experience linked to the research topic (incidents affecting self or 
others close to researchers); 

e personal expertise in relation to the topic (qualification, experience). 


To give an account of the way personal beliefs, interests, experience and 
expertise might have a bearing on findings, the researcher is likely to need to 
draw upon personal details about his or her: 
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e social background (class, family, environment); 
e age; 

© sex; 

e ethnicity; 

e education and qualifications; 

e work experience and skills. 


Access to fieldwork settings 


Gaining access to people, places and events is a crucial part of successful ethno- 
graphic research. For field researchers this access cannot be taken for granted. 
Researchers need to set about gaining access, and to do this they need to engage 
in negotiations that have political, ethical and practical implications. 


Naturalism 


In order to produce a pure and detailed description, ethnographers will wish to 
avoid disrupting the situation by their very presence as observers in the field. 
They will wish to preserve the natural state of affairs. And this is why natural- 
ism is a key concern of ethnography. The ethnographer’s concern with natur- 
alism derives from the wish to study things in their natural state - undisturbed 
by the intrusion of research tools or the disruption of experimental designs. 
Going ‘into the field’ to witness events at first hand in their natural habitat lies 
at the very heart of what it means to do ethnography. 


Link up with Participant observation, p. 206 A 
Covert research 


It is sometimes possible for an ethnographer to conduct fieldwork without 
people knowing that they have a researcher in their midst. The researcher can 
blend into the background by adopting a role that fits in with the normal 
situation. The researcher can go undercover and carry out the research in a 
clandestine manner. This has benefits as far as ethnography is concerned 
because (1) it preserves the naturalness of the setting; and (2) it generally 
sidesteps any need to get authorization to conduct the research. However, it 
gives rise to a different type of problem. The decision to undertake covert 
research means that the researcher cannot also have ‘informed consent’ on the 
part of the subjects of his or her research, and this raises substantial ethical 
problems (see Humphreys 1970). 
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Gatekeepers 


Ethnographers do not always work in a covert manner. Indeed, it is fair to say 
that most ethnographies actually involve some degree of openness about the 
role of the researcher; that is, overt research. When this is the case, the ethical 
problems cease to be the sole concern. Added to this, ethnographers are faced 
with the possibility that their explicit research role might disrupt the natural- 
ness of the situation. Most of all, though, an overt research role will most 
probably bring with it a call for the researcher to make contact with ‘gate- 
keepers’ who can help the researcher with the vital business of gaining access 
to the necessary fieldwork settings. 

Identifying key people who can grant permission, and successfully negotiat- 
ing with them for access to people, places and events, are often a prerequisite 
without which the fieldwork cannot begin. In informal settings, such sponsors 
act as guarantors who vouch for the bona fide status of researcher. They use 
their informal status and relationship with subjects as a currency facilitating 
both contact and trust between researcher and subject or group (e.g. Whyte 
[1943] 1981; Polsky 1967; Hammersley and Atkinson 2007; O’Reilly 2009; 
Murchison 2010). 

Sponsors and gatekeepers, however, cannot be disregarded once their initial 
approval has been obtained. In reality, they exercise continued influence over 
the nature of the research. Their influence persists in terms of both: 


e the kind of initial agreement established about access (influencing the 
nature of eventual research findings); and 

e the need for access to people and places that were not foreseen at the start 
of the research. 


The influence of such gatekeepers, then, goes beyond a simple granting or 
denial of contact with research subjects. As Burgess (1984) argues, access is a 
continual process in fieldwork research. It is a process because research is gener- 
ally conducted over a period of time and it is a process, perhaps more import- 
antly, because access needs to be sought to new people, places and events as 
new lines of enquiry become incorporated in the research. Since field research, 
especially ethnographic work, tends to evolve and develop in terms of its focus 
of enquiry, it is unlikely that the researcher could state at the outset of research 
the precise nature and extent of access that will be required. Access, in the 
sense of permission from a gatekeeper, is necessarily renewable and renegoti- 
able, and should be viewed as an ‘access relationship’ rather than a one-off 
event. 

Gatekeepers’ authority to grant or withhold access varies, of course, depend- 
ing on the degree to which the people, places and events being studied are part 
of a formal environment. Whereas in the educational context headteachers 
have formal jurisdiction over entry to school premises and can withhold 
or grant access to teachers and pupils, in less formal settings the role of the 
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gatekeeper is different. As Whyte illustrated so graphically in his research 
(Whyte [1943] 1981), the role of gatekeeper can become more akin to a guar- 
antor for the bona fide status of the researcher. Whyte’s gatekeeper, ‘Doc’, had 
no formal authority to grant or deny access; he used his status with the group 
to open up the possibility of Whyte making contact with the young men in 
question, and acted as a sponsor/guarantor to enable some trust relationship 
to develop. This is an important point. As Hammersley and Atkinson (2007) 
argue, access is not simply a matter of physical presence or absence. It is far 
more than a matter of granting or withholding of permission for research to be 
conducted. Worthwhile access to people, places and events involves, as well, 
an opening up to allow the researcher into the world of the persons involved. 
Being present is one hurdle, but ‘access’ also involves being trusted with 
insights and insider knowledge. 


Advantages of ethnography 


e Direct observation. As a research strategy it is based on direct observation via 
fieldwork, rather than relying on second-hand data or statements made by 
research subjects. 

e Empirical. It is essentially grounded in empirical research involving direct 
contact with relevant people and places. 

e Links with theory. It can be used as a means for developing theory, and it can 
also be used for testing theories. 

e Detailed data. It provides data which are relatively rich in depth and detail. 
Potentially, it can deal with intricate and subtle realities. 

e Holistic. Ethnography aspires to holistic explanations which focus on pro- 
cesses and relationships that lie behind the surface events. Potentially, it 
puts things in context rather than abstracting specific aspects in isolation. 

e Contrast and comparison. There is an element of contrast and comparison 
built into ethnographic research in the way the distinct culture or events 
being studied are ‘anthropologically strange’ — different from other cultures 
or events which the researcher and his or her audience to some degree share. 

e Actors’ perceptions. Ethnographic research is particularly well suited to deal- 
ing with the way members of a culture see events — as seen through their 
eyes. It describes and explores the ‘actors’ perceptions’. 

e Self-awareness. It has an open and explicit awareness of the role of the 
researcher’s self in the choice of topic, process of research and construction 
of the findings/conclusions. It acknowledges the inherent reflexivity of 
social knowledge. 

e Ecological validity. It is strong in terms of ecological validity, to the extent that 
the act of researching should have relatively little impact on the setting - 
retaining things in their ‘natural’ form. 
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Disadvantages of ethnography 


e Tensions within the approach. There is a tension within the realms of eth- 
nography stemming from its twin concerns with naturalism and reflexivity. 
Ethnographies generally attempt to accommodate an internal contradic- 
tion between (1) ‘realist’ aspirations to provide full and detailed descriptions 
of events or cultures as they naturally exist, and (2) a ‘relativist’ awareness of 
the reflexive nature of social knowledge and the inevitable influence of the 
researcher’s ‘self’ on the whole research endeavour. Depending on how this 
tension is accommodated and the degree to which the ethnography adopts a 
‘realist’ or a ‘relativist’ position, certain other weaknesses can exist. 

e Stand-alone descriptions. Ethnographic research has the potential to produce 
an array of ‘pictures’ which coexist, but which tend to remain as separate, 
isolated stories. There is arguably an in-built tendency to scatter these 
building-block pictures at random, rather than erecting a structure where 
each is a brick which serves as a base for further bricks to be layered upon it 
moving ever upwards. To avoid this, ethnographies need to be guided by (or 
to) a coherent theoretical framework. 

e Story-telling. There is the potential to provide detailed descriptive accounts 
at the expense of developing an analytic insight or contributing to a theor- 
etical position. The story-telling can become the sole purpose, leaving a 
research product which is a-theoretical, non-analytic and non-critical. 

e Reliability. There is a potential weakness of poor reliability and little pro- 
spect of generalizing from the ethnographic account of the culture or event. 

e Ethics. There is also a greater potential than with most other approaches to 
face ethical problems associated with intrusions upon privacy and with 
gaining informed consent from research subjects. 

e Access. Ethnographic research can pose particularly acute difficulties of 
gaining access to settings, access which avoids disrupting the naturalness 
of the setting. 

e Insider knowledge, rather than being a straight advantage, can also cause a 
‘blind spot’, obscuring a vision of ‘the obvious’. 
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Checklist for ethnographic research 


When undertaking ethnographic research you should feel confident about 
answering ‘yes’ to the following questions: 


Have lifestyles and meanings been treated as the main 
interest of the research? 


Has a holistic approach been adopted, linking all 
significant contributing factors? 


Has the research involved a substantial time ‘in the 
field’ collecting empirical data? 


Has access to the fieldwork settings been negotiated 
through relevant ‘gatekeepers’? 


Has the research been conducted ethically with due 
respect to the rights of those being studied? 


Has the research managed to avoid disrupting the 
naturalness of the settings? 


Does the research involve an account of the role of the 
researcher’s ‘self’? 


Is there a description of how the ethnography builds on 
(or contradicts) existing explanations of the phenomenon? 


Are the findings compared and contrasted with other 
ethnographies on similar topics? 


Has due acknowledgement been given to the fact that the 
ethnography is a researcher’s construction rather than a literal 
description of the situation? 


© M. Denscombe, The Good Research Guide. Open University Press. 


PHENOMENOLOGY 


What is the phenomenological approach? e Experience e Everyday 
world e Seeing things through the eyes of others e The social 
construction of reality e Multiple realities e Description e The 
suspension of common-sense beliefs e Members’ accounts e Types 


of phenomenology: underlying essences or actual experiences e 
Advantages of phenomenology e Disadvantages of phenomenology: 
underlying essences or actual experiences e Checklist for 
phenomenology 


What is the phenomenological approach? 


As an approach to social research, phenomenology is sometimes presented as 
an alternative to positivism. When a simple dichotomy is called for, phenom- 
enology has been useful for some writers as an umbrella term covering styles of 
research that do not rely on measurement, statistics or other things generally 
associated with the scientific method. In direct contrast to positivism, it is seen 
as an approach that emphasizes: 


subjectivity (rather than objectivity); 
description (more than analysis); 
interpretation (rather than measurement); 
agency (rather than structure). 


Its credentials as an alternative to positivism are further reinforced by the fact 
that phenomenological research generally deals with: 
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e people’s perceptions or meanings; 
e people’s attitudes and beliefs; 
e people’s feelings and emotions. 


Link up with Frequently asked questions, ‘What is positivism?’, p. 326 A 

"ee 
Such a thumbnail portrait of phenomenology serves to set the scene and pro- 
vide an initial view of its direction as a research strategy. It helps to explain 
why a phenomenological approach has proved useful for researchers in areas 
such as health, education and business who want to understand the thinking 
of patients, pupils and employees (van Manen 1990; Crotty 1996). And it gives 
a clue to why phenomenology is associated with humanistic research using 
qualitative methodologies —- approaches that place special emphasis on the 
individual’s views and personal experiences. 


Phenomenology is an approach that focuses on how life is experienced 


It is not primarily concerned with explaining the causes of things but tries, 
instead, to provide a description of how things are experienced at first hand by 
those involved. The phenomenological investigation of something like ‘home- 
lessness’, for instance, would focus on the experience of being homeless. It 
might be interested in trying to get at the essence of what it means to be 
homeless. It might try to understand homelessness from the point of view of 
those who are themselves homeless and try to describe how they see things, 
how they understand the situation, how they interpret events. What it would 
not try to do is measure the extent of homelessness, or explain the causes of 
homelessness. Though both might be worthwhile endeavours, neither would 
be at the heart of phenomenological research. 


Experience 


Phenomenology is concerned, first and foremost, with human experience — 
something denoted by the term ‘phenomenology’ itself. A phenomenon is 
something that stands in need of explanation; something of which we are 
aware but something that, as yet, remains known to us only in terms of how it 
appears to us directly through our senses. A phenomenological approach to 
research, following from this, concentrates its efforts on the kind of human 
experiences that are pure, basic and raw in the sense that they have not (yet) 
been subjected to processes of analysis and theorizing. In contrast to other 
approaches to research that rely on processes of categorizing things, abstract- 
ing them, quantifying them and theorizing about them, phenomenology 
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prefers to concentrate its efforts on getting a clear picture of the ‘things in 
themselves’ — the things as directly experienced by people. 


Everyday world 


Phenomenology is also characterized by a particular interest in the basics 
of social existence. This stems from a philosophical concern with the nature 
of ‘Being-in-the-world’ (Heidegger 1962) and the lived experience of human 
beings within the ‘life-world’ (Husserl 1970). In practice, this translates into 
special importance being attached to the routine and ordinary features of 
social life, and to questions about how people manage to ‘do’ the everyday 
things on which social life depends. Sometimes this can involve quite funda- 
mental things associated with the way people experience their lives, for 
example, coming to terms with the death of a loved one (Seymour 1999), 
major life transitions and identity changes (Smith et al. 2009), or the aware- 
ness of what it means to be ‘different’ from others (Moustakas 1992). On other 
occasions phenomenological studies have looked at more mundane things 
such as daydreaming (Morley 1998) and complaining about chronic pain 
(Kugelmann 1999). Such topics might draw on normal, routine facets of the 
everyday world around us but, from the phenomenological perspective, this 
does not make them trivial or inconsequential. 


Seeing things through the eyes of others 


When dealing with the way people experience facets of their lives, phenom- 
enology stresses the need to present matters as closely as possible to the 
way that those concerned understand them. The phenomenologist’s task, 
in the first instance, is not to interpret the experiences of those concerned, 
not to analyse them or repackage them in some form. The task is to present 
the experiences in a way that is faithful to the original. This entails the ability 
to see things through the eyes of others, to understand things in the way 
that they understand things and to provide a description of matters that 
adequately portrays how the group in question experiences the situation. 
This has some significant implications for social research. First, it places the 
ideas and reasoning of the group being studied at the core of the investigation. 
The research necessarily revolves around people and their lives, rather than 
systems, aggregates and trends operating at a high level of abstraction. Second, 
it elevates the importance of people’s thinking in terms of the research. Rather 
than being treated as something that is the crude foundation from which to 
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build more sophisticated explanations and theories about the experiences, 
that thinking becomes the topic of investigation in its own right. Third, 
people’s everyday thinking is given credibility and respected in its own right as 
valid. It is not necessarily treated as a less rational than or inferior to the 
‘scientific’ thinking of the social researcher, but is considered as rational in its 
own terms of reference. For these reasons, phenomenology has an affinity with 
humanistic perspectives on research that are keen to accord normal people and their 
own everyday reasoning higher status in research. 


The social construction of reality 


Phenomenology is particularly interested in how social life is constructed 
by those who participate in it, and it makes two points about this. First, 
it regards people as creative interpreters of events who, through their actions 
and interpretations, literally make sense of their worlds. From the perspective 
of the phenomenologist, people do not passively obey a set of social rules; 
nor do they slot into an external social structure; nor do they simply respond 
to their internal physiological drives or psychological dispositions. They 
are viewed instead as ‘agents’ who interpret their experiences and who actively 
create an order to their existence. As an approach to understanding the 
social world this sets phenomenology apart from any belief that things are 
preordained, that there is a reality to social life that exists independently from 
the way people experience it or that the way we experience and understand 
things is structured by the way human minds are programmed to perceive 
things. 


Good practice: making sense of reality 


Phenomenological research should focus on the ways in which people inter- 
pret events and, literally, make sense of their personal experiences. 


Second, it acknowledges that the processes of interpreting sights and sounds 
into meaningful events are not unique to each individual. Necessarily, they 
must be shared with others who live in the group or community (Berger and 
Luckmann 1967). If they were not, people would find it virtually impossible to 
interact with one another, since they could be operating on quite different 
understandings about what is going on. They could be living in different 
worlds, unable to communicate and unable to grasp the implications of other 
people’s actions. Without some shared basis for interpreting their experiences 
there would be no way of knowing what others were doing or what their 
intentions were. There would, effectively, be no basis for social life. 
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Multiple realities 


When the social world is seen as ‘socially constructed’ it opens up the possibil- 
ity that different groups of people might ‘see things differently’. There is the 
prospect of alternative realities — realities that vary from situation to situation, 
culture to culture. In this respect phenomenology stands in stark contrast to 
positivistic approaches to social research that are premised on the assumption 
of one reality. Whereas positivistic approaches look for explanations that fit 
one universal reality, phenomenological approaches tend to live with, even 
celebrate, the possibility of multiple realities. Reflecting the fact that the world 
as experienced by living humans is something that is created through the way 
they interpret and give meaning to their experiences, phenomenology rejects 
the notion that there is one universal reality and accepts, instead, that things 
can be seen in different ways by different people at different times in different 
circumstances, and that each alternative version needs to be recognized as 
being valid in its own right. 


/\ Caution: multiple realities 


This phenomenological position certainly allows for competing ver- 
sions of what things are, how they work and what they mean. It implies 
that there might not be one ‘right’ explanation and that different theor- 
ies might be equally valid. It moves researchers away from a mind set 
where one theory is correct and others are inadequate or wrong. It does 
not imply, however, that there are as many social realities as there are 
individuals — each interpreting his or her world in their own unique 
way. As has been stressed, phenomenology does not treat interpret- 
ations of the social world as totally individual things. Necessarily, they 
are shared between groups, cultures and societies, and it is only at 
these levels that phenomenology recognizes the possibility of there 
being multiple realities. 


Description 


When it comes to the matter of how phenomenologists actually do their 
research, a key characteristic of the approach is its emphasis on describing 
authentic experiences. Rather than directing their attention to explanations 
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and analyses of experiences in an attempt to discover why they occurred, 
phenomenologists focus instead on trying to depict the relevant experiences 
in a way that is as faithful to the original as possible. 

To provide a description of the authentic experience, phenomenological 
research first and foremost needs to provide a description that adequately 
covers the complexity of the situation. One of the crucial benefits of a phe- 
nomenological approach is that it deals with things in depth and does not try 
to gloss over the subtleties and complications that are essential parts of many — 
possibly most — aspects of human experience. The description, therefore, must 
be fairly detailed. 

The phenomenological description should also be prepared to recognize 
and include aspects of the experience that appear to be self-contradictory, 
irrational or even bizarre. The researcher’s role is not to act as editor for the 
way people explain their experiences. Nor is it to impose some artificial order 
on the thoughts of those being studied by trying to remedy any apparent 
logical inconsistencies. The social world is rarely neat and phenomenology 
does not involve the attempt to present life experiences as though they are 
entirely coherent. 


Good practice: dealing with complexities and contradictions 


Phenomenological research should involve a detailed description of the 
experience that is being investigated — one that does not gloss over the com- 
plexities and contradictions that inhabit real life. 


The suspension of common-sense beliefs 


A phenomenological approach not only encourages the researcher to provide 
a detailed description of experiences, it also advocates the need to do so 
with a minimum reliance on the researcher’s own beliefs, expectations and 
predispositions about the phenomenon under investigation. 

From a phenomenological perspective, researchers are part and parcel of the 
social world they seek to investigate. Phenomenologists see the social world as 
existing only through the way it is experienced and interpreted by people - 
and this includes researchers as much as anyone else. Researchers, like other 
people, use common-sense assumptions when interpreting events, and this is 
where phenomenology has an important point to make for researchers. 
Researchers need to be aware of the fact that they rely on such everyday com- 
mon sense, and make an effort to minimize the impact of these assumptions. 
They need to stand back from their ordinary, everyday beliefs and question 
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them to see if they are blinkering the researcher’s vision of what is happening. 
As best they can, researchers need temporarily to suspend common-sense 
beliefs. For the purposes of being able to provide a ‘pure’ description, 
researchers need to approach things without predispositions based on events 
in the past, without suppositions drawn from existing theories about the phe- 
nomenon being studied and without using their everyday common-sense 
assumptions. Such things need to be put to one side for the moment, ‘brack- 
eted off’ so that researchers are able to describe things not through their own 
eyes but through the eyes of those whose experiences are being described. 

One way of ‘bracketing off’ presuppositions is to adopt the stance of ‘the 
stranger’ (Schutz 1962). The value of this is that it allows the researcher to see 
things that would otherwise be hidden from view - things that are so normal 
and obvious that they would not come to the attention of the researcher as 
something significant. A stranger is naive about how things work, and needs to 
figure them out from first principles before he or she can begin to operate as a 
competent member of the society. A ‘stranger’ is faced with the prospect of 
understanding how members of the society make sense of events. By adopting 
the stance of the stranger, then, the researcher is best able to see things for 
what they are, uncluttered by assumptions that form part of everyday thinking 
about those things. 


Good practice: suspending common-sense beliefs 


Researchers who use a phenomenological approach need to be explicit about 
their own ways of making sense of the world and, in order to get a clear view of 
how others see the world, they need to suspend (or bracket off) their own 
beliefs temporarily for the purposes of research. 


Members’ accounts 


Data collection by phenomenology tends to rely on tape-recorded interviews. 
The interviews are conducted with members of the particular group whose 
experiences are being investigated. These members can provide insights into 
the thinking of the group and, through in-depth interviews, their experiences 
and their reasoning can be described and explained to the researcher in a way 
that allows the researcher to see things from the member’s point of view 
(Denscombe 1983). 

The process of interviewing is valuable for phenomenologists for a number 
of reasons. First, it provides the possibility of exploring matters in depth. In 
phenomenological research, interviews tend to be relatively long (in the 


100 THE GOOD RESEARCH GUIDE 


region of one hour, often more) so that there is plenty of time to delve deeply 
into issues as they arise. Second, interviews allow the interviewee to raise issues 
that he or she feels are important. This helps the phenomenologist’s investiga- 
tion by highlighting things that matter to the person being interviewed. It 
helps to ‘see things from the member’s point of view’. To encourage this, 
phenomenological researchers tend to use relatively ‘unstructured’ interviews. 
Rather than the interview being conducted using a series of questions devised 
by the researcher, unstructured interviews allow plenty of scope for inter- 
viewees to move the discussion to areas that they regard as significant. Third, 
the interview process gives an opportunity for interviewees to provide an 
‘account’ of their experiences. It is their version, spoken in their own words. It 
allows them both to describe the situation as they see it and to provide some 
justification or rationale, again from their point of view. Fourth, interviews 
allow the researcher the opportunity to check that he or she is understanding 
the interviewee correctly. As a normal part of the interview process the 
researcher can take special care to make sure that what he or she is ‘hearing’ is 
really what the interviewee is trying to put across, not a partial or mistaken 
interpretation resulting from the researcher’s common-sense assumptions or 
presuppositions. 


Link up with Interviews, Chapter 10 . 
"ee 


Types of phenomenology: underlying essences or 
actual experiences 


Over time, different versions of phenomenology have developed and there 
are sometimes disagreements among those who would call themselves ‘phe- 
nomenologists’ about which is the ‘true’ version. The social researcher who 
is thinking about adopting a phenomenological approach should be aware 
that such differences of opinion exist. Although many versions exist, it is 
probably most straightforward to characterize phenomenology as consisting 
of two main types. One version derives from the European tradition of 
thought and the other, the ‘new phenomenology’, has a North American 
origin (Crotty 1996). 

The European version is more steeped in the discipline of philosophy, and 
can lay claims to being the original version, since it owes much to the 
founding father of phenomenology, Edmund Husserl. His ‘transcendental 
phenomenology’ (Husserl 1931) approaches the study of human experience 
with the aim of discovering underlying, fundamental aspects of experience — 
features that are universal and that lie at the very heart of human experience. 
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Though differing somewhat, the ‘existential phenomenology’ of Jean-Paul 
Sartre (1956) and the ‘hermeneutic phenomenology’ of Martin Heidegger 
(1962) draw on similar European philosophical roots and have a crucial thing 
in common. They share a concern with investigating the essence of human 
experience. 

The kinds of experience dealt with by these writers tend to revolve around 
fundamental philosophical questions about ‘being-in-the-world’ and similar 
issues relating to ‘the meaning of (everyday) life’. However, this form of phe- 
nomenology can operate at a more mundane level. For example, it could 
address something as topical and down-to-earth as bullying in schools. Based 
on this tradition of phenomenology, the research would be interested in the 
experience of being bullied and, in particular, it would focus on the essential 
features of that experience. It would try to identify the core components of the 
notion of bullying by investigating the experiences of those who have found 
themselves the victim of bullying. It would not focus on the extent of bullying 
in schools, nor would it try to explain the causes of bullying. Its focus would be 
on the essence of the experience of being bullied. 

Importantly, though, the European tradition of phenomenology stresses 
that such essences exist beyond the realms of individual, personal experiences 
of specific events. While such individual instances might be the starting point 
for investigation, the purpose is always to use them as a means for getting a 
clearer picture of the essential qualities of the experience that exist at a general 
level. In relation to the example of bullying, individual instances of bullying 
would be of interest only to the extent that they could be used to help the 
identification of the broader, essential qualities of that experience: they would 
be valuable as means towards an end, but would not constitute an end in 
themselves as far as research is concerned. 

Contrasting with this tradition, there is the North American version of phe- 
nomenology which is more commonly linked to the disciplines of sociology, 
psychology, education, business studies and health studies. It emanates from 
the ‘social phenomenology’ of Alfred Schutz (1962, 1967). Schutz was primar- 
ily interested in the mental processes through which humans make sense of 
the many things they experience. His early work focused on the way that, in 
order to make sense of the social world, people interpret things through seeing 
them as specific instances of more general categories — a process that involves 
what he referred to as ‘typifications’. Within the realms of phenomenology 
this shift in focus had important repercussions. Indeed, through his focus on 
interpretation of the social world, Schutz’s work actually spawned a rather 
different kind of phenomenology. This kind of phenomenology is less con- 
cerned with revealing the essence of experience, and more concerned with 
describing the ways in which humans give meaning to their experiences. Here 
lies the defining characteristic of the North American approach: its concern 
with the ways people interpret social phenomena. 

This form of phenomenology retains a concern with things like experience, 
an interest in everyday life and suspending common-sense assumptions in 
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order to provide a description that is true to the way those involved experience 
things. However, it also incorporates elements of pragmatism and social inter- 
actionism into the approach — things that make it controversial in two ways 
as far as the European phenomenological tradition is concerned. First, the 
North American phenomenology is comfortable in describing what is being 
experienced (for example, in relation to something such as homelessness), 
rather than attempting to uncover the essence of what is meant by the 
term. Second, and linked with this, it also places some value on describing 
individuals’ experiences for their own sake. The new phenomenology might 
well investigate homelessness by asking what being homeless means to those 
who are homeless, what events and emotions they have experienced as a 
consequence of being homeless and what it means to them. The experiences 
of the individual are taken as significant data in their own right, not some- 
thing to be put to one side in order to identify the universal essence of the 
phenomenon. 


Good practice: clarity of purpose 


In practice, it is not always easy to separate the two traditions of phenomen- 
ology. Ideas from one area are imported to the other, not always reflecting the 
original very faithfully. Such blurring of the boundaries complicates life for 
project researchers. This should not, though, deter a researcher from choosing 
this approach, since many aspects of social research operate in such con- 
tested domains. It does, however, highlight the need for the researcher to be 
very clear about his/her specific aims when investigating the phenomena of 
human experience. 


Advantages of phenomenology 


e Suited to small-scale research. Phenomenological research generally relies on 
in-depth interviews and does not call for technologically sophisticated or 
expensive equipment for the purposes of data collection and analysis. 
Coupled with this, it is often undertaken in specific localities such as hos- 
pitals, schools or industrial plants. In both respects, it lends itself to small- 
scale research where the budget is low and the main resource is the 
researcher himself/herself. 

e The description of experiences can tell an interesting story. There is an inherent 
potential within (new) phenomenology to describe experiences in a way 
that is immediately accessible and interesting to a wide range of readers. By 
unfolding the events and laying bare the feelings experienced by people, 
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the research is likely to attract a relatively wide readership. It deals with 
everyday life and people can generally relate to this. 

e Offers the prospect of authentic accounts of complex phenomena. The social 
world is complex and rarely straightforward. A phenomenological approach 
allows the researcher to deal with that complexity. It scratches beneath the 
superficial aspects of social reality. It calls for the researcher to delve into 
phenomena in depth and to provide descriptions that are detailed enough 
to reflect the complexity of the social world. 

e A humanistic style of research. There is a respect for people built into the 
phenomenological approach. It carries an aura of humanism and, in its 
efforts to base its enquiry on the lived experiences of people in the everyday 
world, it represents a style of research that is far removed from any high- 
minded, abstract theorizing. In effect, the researcher needs to be close to the 
objects of study. 


Disadvantages of phenomenology 


e Lacks scientific rigour. The emphasis of phenomenology on subjectivity, 
description and interpretation contrasts with the scientific emphasis on 
objectivity, analysis and measurement. Although phenomenology is self- 
consciously non-positivist - and proud to be so — there is the danger that 
this can be turned against it and be treated as a weakness rather than a 
strength by those who do not share its stance. 

e Associated with description and no analysis. The importance attached to pro- 
viding a detailed and accurate description of the events and experiences 
being studied can lead to accusations that phenomenology does nothing 
but provide descriptions. This might not be warranted, in particular where 
the phenomenologist goes on to develop explanations based on the descrip- 
tive material. There is, none the less, the danger that those who are not 
sympathetic to phenomenology might seize the opportunity to criticize it 
for its primary focus on description. 

e Generalizations from phenomenological studies. Phenomenological research 
does not normally involve large numbers or instances of the phenomenon 
being studied. This will always raise questions about the representativeness 
of the data and how far it is justifiable to generalize from the findings. 
Phenomenologists, for their part, might not regard such a concern as rele- 
vant to their work, but many other kinds of researchers will be concerned 
with this issue. 

e Attention to the mundane features of life. For phenomenologists the study 
of routine aspects of everyday life occurs because it is fundamental for 
understanding the nature of the social world. For others it might be 
(mis)interpreted as dealing with things that are mundane, trivial and 
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relatively unimportant compared with the big issues of the day in the 
spheres of social policy, international relations, economic progress, and 
the like. 

e Feasibility of suspending common sense. In principle, suspending presupposi- 
tions about the way things work might seem a reasonable way of trying to 
get a clearer view of them. However, it is doubtful indeed if it is ever possible 
to rid ourselves entirely of such presuppositions. Socialization and the use 
of language make it impossible. What can be done, though, is to be reflect- 
ive and self-conscious about the way perceptions are shaped by things like 
common sense and then to try to moderate their impact. 
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Checklist for phenomenology 


When undertaking phenomenological research you should feel confident 
about answering ‘yes’ to the following questions: 


Is the research concerned with one or more of the following: 


Looking for the essential features of a phenomenon? 
Describing the content and nature of lived experiences? 
Investigating the everyday rules by which 

experience is structured? 


Does the research take into consideration: 


e The social construction of reality? 
e Multiple realities? 


Have efforts been made to bracket off common-sense 
assumptions and take account of researchers’ presuppositions 
in order to provide a ‘pure’ description? 


Does the research provide a description that adequately 
deals with the complexity, subtlety and contradictions of the 
phenomenon? 


Does the description authentically represent the way 
things are seen by those whose experiences are being studied? 


Does the research involve the suitable use of interviews 
and transcripts for checking members’ meanings and intentions? 


© M. Denscombe, The Good Research Guide. Open University Press. 


GROUNDED THEORY 


What is the grounded theory approach? e When is the grounded theory 
approach useful? e Methods of data collection e Initial ideas and 
concepts: the starting point e Initial sites for fieldwork e Subsequent 
sites for fieldwork: theoretical sampling e Analysing the data 


e Completing the research (theoretical saturation) e Theories and 
grounded research e Developments in grounded theory e Advantages 
of the grounded theory approach e Disadvantages of the grounded 
theory approach e Checklist for the grounded theory approach 


The grounded theory approach has become a popular choice of methodology 
among social researchers in recent times. In particular, it has been adopted by 
those engaged in small-scale projects using qualitative data for the study of 
human interaction, and by those whose research is exploratory and focused on 
particular settings. ‘Grounded theory’ is cited frequently in such research - not 
least because it has come to provide a well recognized, authoritative rationale 
for the adoption of an approach that does not necessarily involve statistical 
analysis, quantitative data or the quest for representative samples. 


What is the grounded theory approach? 


The approach originated with the work of Barney Glaser and Anselm Strauss 
and, in particular, their book The Discovery of Grounded Theory, which was 
published in 1967. Since that time, the notion of grounded theory has come to 
mean slightly different things to different people. There has been a tendency 
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for researchers to ‘adopt and adapt’ grounded theory and to use it selectively 
for their own purposes. There are, though, certain basic ideas associated with 
the grounded theory approach that remain fairly constant, and these are out- 
lined in this chapter. They draw principally on the writings of Glaser and 
Strauss themselves and the main focus is on those components of the 
grounded theory approach that mark it out as distinctive and constitute its 
core components as an approach to social research. 


What is the grounded theory approach? 


It is an approach dedicated to generating theories. In this 
sense it contrasts with approaches concerned with testing 
theories, and is different from research whose main purpose 
is to provide descriptive accounts of the subject matter. 


Theory 


It is an approach that emphasizes the importance of 
empirical fieldwork and the need to link any explanations 
very closely to what happens in practical situations in ‘the 
real world’. It is different, here, from approaches based on 
‘armchair theorizing’ or explanations that are thought up in 
the abstract as a neat system of ideas, and then afterwards 
checked to see if they work in reality. 


Grounded 


Theories should be ‘grounded’ in empirical research 


The approach directly challenges the value of theorizing at a high level of 
abstraction and then, subsequently, doing some empirical work to see if the 
theory actually works. It is much better to develop the theories on the basis of 
empirical research and gradually build up general theories that emerge from 
the data. 

The insistence that theories should be ‘grounded’ brings with it the idea that 
researchers should be engaged with fieldwork as the fundamental part of the 
work they do. When adopting the grounded theory approach, the researcher 
should undertake data collection in the field, not only as the starting point of 
the research but throughout the course of the research as well. 


Theories should be generated by a systematic analysis of the data 


Glaser and Strauss were keen to stress that an emphasis on collecting data in 
the field does not equate their approach with those whose purpose is to amass 
as much detail as possible about particular situations and then ‘let the data 
speak for themselves’. It differs in this respect from certain kinds of ethno- 
graphic research. Always high on the agenda for grounded theory research is a 
concerted effort to analyse the data and to generate theories from the data. And 
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grounded theory is characterized by the specific way in which it approaches 
the business of analysing the data. Concepts and theories are developed out of 
the data through a persistent process of comparing the ideas with existing 
data, and improving the emerging concepts and theories by checking them 
against new data collected specifically for the purpose. 


The selection of instances to be included in the research reflects the developing 
nature of the theory and cannot be predicted at the start 


The instances to be included follow a trail of discovery — like a detective follows 
a ‘lead’. Each new phase of the investigation reflects what has been discovered 
so far, with new angles of investigation and new avenues of enquiry to be 
explored. So, in the spirit of ‘grounded theory’ it is neither feasible nor desir- 
able for the researcher to identify prior to the start exactly who or what will be 
included in the sample. This, of course, runs right against the grain as far as 
conventional social research practice is concerned. Normally, the social 
researcher would be expected to have a very clear vision of the intended sam- 
ple, chosen on the basis of criteria linked to the ideas and theories being tested 
by the research. 


Researchers should start out with an ‘open mind’ 


In marked contrast to the researcher who sets out to test a theory, the 
researcher who follows the principles of grounded theory needs to approach the 
topic without a rigid set of ideas that shape what they focus upon during 
the investigation. Rather than basing an investigation upon whether certain 
theories do or do not work, the researcher embarks on a voyage of discovery. 

This should not be seen to imply that a good social researcher starts out with 
a ‘blank’ mind. 


To be sure, one goes out and studies an area with a particular . . . perspec- 
tive, and with a focus, a general question or a problem in mind. But the 
researcher can (and we believe should) also study an area without any 
preconceived theory that dictates, prior to the research, ‘relevancies’ in 
concepts and hypotheses. 

(Glaser and Strauss 1967: 33) 


So an open mind is not a blank mind on a subject. It is informed about an 
area, even quite aware of previous theories that might apply, but does not 
approach the analysis of data using preordained ways of seeing things. It 
avoids using previous theories and concepts to make sense of the data and 
thus is open to discovering new factors of relevance to an explanation of 
that area. 
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Theories should be useful at a practical level and meaningful to those 
‘on the ground’ 


The grounded theory approach has its roots in pragmatism, whose guiding 
philosophy is clearly acknowledged by Glaser and Strauss. Pragmatism places 
great emphasis on the ‘practical’ rather than the ‘abstract’ when it comes to 
issues of knowledge and truth and it operates on the premise that the value of 
any theory can only be gauged by how well it addresses real practical needs 
and how well it works in practice. 


Good practice: grounded theory should make sense to those whose actions 
it explains 

Grounded theory acknowledges its pragmatist philosophical heritage in insist- 
ing that a good theory is one that will be practically useful in the course of daily 
events, not only to the social scientists, but also to laymen. In a sense, a test of 
a good theory is whether or not it works ‘on the ground’ (Locke 2001: 59). 


When is the grounded theory approach useful? 


The grounded theory approach does not purport to be the only proper way of 
conducting social research. It is but one approach among many. What it does 
claim, however, is that it offers an approach that is well suited to the needs of 
four kinds of research: 


qualitative research; 
exploratory research; 

studies of human interaction; 
small-scale research. 


PUNE 


Qualitative research 


Although in their original exposition of grounded theory Glaser and Strauss 
claimed that the approach could be applied to quantitative data as well as 
qualitative data, and despite subsequent claims that ‘It transcends specific data 
collection methods’ (Glaser 1978: 6), the grounded theory approach has 
become firmly associated over time with qualitative research. The nature of 
Glaser and Strauss’s own research effectively set the tenor for the kind of 
research that has come to be associated with the grounded theory approach 
and by far the most common use of the grounded theory approach by other 
researchers has been with studies using qualitative data. 
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Exploratory research 


Because there is an emphasis on discovery and because there is a stress on the 
need to approach investigations without being blinkered by the concepts and 
theories of previous research, the grounded theory approach fits neatly with 
the needs of researchers who are setting out to explore new territory in terms of 
either the subject matter of their investigation or the extent to which relevant 
theories have already been developed. As Goulding (2002: 55) notes, ‘Usually 
researchers adopt grounded theory when the topic of interest has been rela- 
tively ignored in the literature or has been given only superficial attention.’ 


Studies of human interaction 


The grounded theory approach is appropriate for social research that focuses 
on human interaction, particularly where the researcher wishes to investigate 


e practical activity and routine situations; 
e the participants’ points of view. 


This reflects the roots of grounded theory within an approach called symbolic 
interactionism. This is a form of social research that focuses on the way that 
participants in social settings make sense of situations and events through the 
use of symbolic actions. 


Small-scale research 


Grounded theory’s affinity with qualitative research, its desire to generate 
explanations from the study of particular instances, its need for detailed data 
about activities and practice and its value for exploratory research, combine to 
make it an approach that is well suited to small-scale research conducted by 
individual researchers operating within the constraints of a tight budget. 


Methods of data collection 


There is not a particular method of data collection that is claimed to be unique 
to grounded theory. Indeed, as Strauss (1987: 1) has indicated, ‘Very diverse 
materials (interviews, transcripts of meetings, court proceedings; field observa- 
tions; other documents, like diaries and letters; questionnaire answers; census 
statistics; etc.) provide indispensable data for social research.’ 

Having made this point, there are certain methods that lend themselves 
better than others to use within a grounded theory approach. These are 
methods that allow the collection of data in a ‘raw’ state - not unduly shaped 
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by prior concepts or theories. The point is to generate theories, not to test 
them, and so there is a preference for unstructured interviews rather than 
structured interviews, for the use of open-ended questions in a questionnaire 
rather than fixed-choice answers, and the use of fieldnotes rather than obser- 
vations based on a tick-box schedule. The preference, quite logically, is for the 
use of methods that produce qualitative data that are relatively unstructured. 


Good practice: the use of unstructured data 


Methods should be used that produce unstructured data — normally in the 
form of text or images. Unstructured interviews, open-ended questions in 
questionnaires, official reports, company records, and fieldnotes based on 
observations can all produce suitable data. 


Initial ideas and concepts: the starting point 


The grounded theory approach expects the researcher to start research without 
any fixed ideas about the nature of the setting that is about to be investigated. 
The aim is to approach things ‘with an open mind’. Taken to extremes, this 
could imply that the researcher should approach the topic without being 
influenced by previous theories or other research relating to the area. Even a 
literature review on the subject might be prohibited. In practice, however, the 
adoption of a grounded theory approach has not actually involved such 
extremes and, indeed, it is quite clear in some of the writings of the originators 
of grounded theory that the extreme interpretation was not what they 
intended to mean. A more moderate version of the point quite clearly accepts 
that previous theories and personal experience will have an influence. The 
crucial point is that, whatever the source of ‘what we already know about the 
topic’, the existing knowledge and concepts are to be treated as ‘provisional’ 
and open to question. They are not fixed, they are not necessarily right. They 
are simply a tentative starting point from which to launch the investigation. 


Initial sites for fieldwork 


Researchers who adopt the grounded theory approach need to start their 
investigation by concentrating on a particular situation, event or group. At the 
beginning, all that is required is that this site is ‘relevant’. The criterion for its 
selection need only be that it might reasonably be expected to provide relevant 


112 THE GOOD RESEARCH GUIDE 


information on the situation, event or group the researcher is interested in 
investigating. To illustrate the point, if a researcher wishes to investigate ‘class- 
room control’, he or she could initiate research by observing a small number of 
lessons in one school. In the first instance, the selection of the particular lessons 
and the specific school does not matter too much. When some lessons have 
been observed and various participants interviewed, however, the subsequent 
sites for research will be selected to follow up ideas prompted by the data. From 
this time forward, the selection of sites becomes far more critical for the 
research and far more constrained by the need to fit the lines of enquiry sug- 
gested by the codes, concepts and categories developing from the research. 
Following the example above, sites (schools) might be selected as instances 
where control problems are acknowledged to be rife, and lessons might be 
selected for observation that offer a relevant contrast in terms of the way they 
are routinely organized (history and English lessons in a normal classroom 
being compared with art lessons, sports classes and chemistry laboratory ses- 
sions). But, at the very beginning, the relevance of such different contexts 
might not have been realized. 

The criterion of ‘relevance’ means that the grounded theory approach takes 
a distinct position in its choice of the initial site for fieldwork. It differs from 
the case study approach in the sense that there is no need for researchers using 
the grounded theory approach to get overly concerned about locating their 
initial fieldwork in a setting that can be demonstrated as being ‘representa- 
tive’, ‘typical’ or ‘extreme’ types. 


p9 
Link up with Case studies, Chapter 2 o 
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Subsequent sites for fieldwork: theoretical sampling 


After having selected the initial site for research (based on its likely relevance 
for the topic), subsequent sites are selected according to a process of theoretical 
sampling. This form of sampling is a distinctive feature of the grounded theory 
approach and it involves three key features: 


e Sites to be included in the research are deliberately selected by the 
researcher for what they can contribute to the research. This means that, 
unlike random sampling, theoretical sampling is a form of non-probability 
sampling in which the new sites are consciously selected by the researcher 
because of their particular characteristics. 


p90 
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e Sites are selected because of their relevance to emerging categories and con- 
cepts. They are chosen to allow comparisons and contrast with previous 
sites and they allow the researcher to test out emerging concepts and verify 
the developing theory as the research goes along. As Corbin and Strauss 
(2008: 144) stress: 


In theoretical sampling the researcher is not sampling persons but 
concepts. The researcher is purposely looking for indicators of those 
concepts so that he or she might examine the data to discover how 
concepts vary under different conditions. 


e Researchers following the principles of grounded theory will not want to, or 
be able to, specify at the outset exactly what the sample will include. They 
will not be able to state exactly how large the sample will be nor exactly 
what sites (events or whatever) will be included because the sample emerges, 
reflecting the pursuit of generating theory. The process of research will 
involve the continued selection of units until the research arrives at the 
point of ‘theoretical saturation’ (see below). It is only when the new data seem 
to confirm the analysis rather than add anything new that the sampling 
ceases and the sample size is ‘enough’. Here, again, theoretical sampling 
stands in stark contrast to the conventions that expect researchers to be 
clear at the outset about the size and nature of their intended sample, and 
how it constitutes a representative sub-section of the overall group of things 
that are to be studied. 


Strauss and Corbin (1990) draw our attention to four other features of theor- 
etical sampling that are important to recognize. First, theoretical sampling 
is cumulative. There is a sense of building on the previous instances that 
have been sampled to date, and on building up a strong foundation for 
the concepts and categories that are being refined through the grounded 
research approach. Events, instances or people sampled early in the research 
should be considered as an integral part of the overall sample - a vital 
component in the path of discovery - and not jettisoned as being obsolete, 
irrelevant or wrong. 

Second, theoretical sampling involves an increased depth of focus. Initially, 
the researcher’s aim is to generate a large number of categories related to the 
phenomenon being studied — to keep things as ‘open’ as possible early on. For 
this purpose it is necessary to spread the net wide and gather data on a wide 
range of relevant things. As the work progresses, though, certain initial codes 
and categories will become relegated in importance and not pursued. As things 
move on, the researcher will find himself or herself able to concentrate on a 
smaller number of codes and categories — the ones that emerge as being more 
crucial to the analysis. And, because these ones are more central to the analysis 
and are fewer in number, he or she will be in a position to probe them more 
fully and investigate them in more depth. 

Third, theoretical sampling follows a rationale. Theoretical sampling is 
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not random or haphazard. It is not based on arbitrary choices, whim or 
convenience. It follows a rationale. This rationale is based on the contribution 
that any new additions to the sample might make towards developing and 
refining codes and concepts. The selection, in this sense, is based on theoretical 
relevance. And, at each stage of the sampling, the selection criteria should be 
clear, systematic and consistent with the emerging theory. 

Fourth, theoretical sampling needs to retain some element of flexibility. 
Although the previous three features of theoretical sampling emphasize focus, 
relevance and system — a funnelling of the focus of attention that concentrates 
research on to key components - there also needs to be a degree of flexibility 
built into the selection process. Partly, this is necessary to allow the researcher 
to respond to opportunities that arise during the course of fieldwork. Equally, 
it is vital to permit the exploration of new avenues of investigation that the 
researcher could not have anticipated as being relevant. Without flexibility, 
the path of exploration is likely to become set too firmly in its direction too 
early in the programme of research, curtailing potentially useful lines of 
enquiry and closing off areas of investigation that might have proved to be 
illuminating - possibly crucial — to the eventual explanation of the situation 
being studied. 


$ 


Key point: flexibility and theoretical sampling 


What makes theoretical sampling different from conventional methods of 
sampling is that it is responsive to the data rather than established before 
the research begins. This responsive approach makes sampling open and 
flexible . . . With theoretical sampling, there is a flexibility to go where 
analysis indicates would be the most fruitful place to collect more data 
that will answer the questions that arise during analysis. 

(Corbin and Strauss 2008: 144, 145) 


Analysing the data 


Analysis involves the separation of things into their component parts. More 
specifically, it involves the study of complex things in order to identify their 
basic elements. It calls on the researcher to discover the key components or 
general principles underlying a particular phenomenon so that these can be 
used to provide a clearer understanding of that thing. The analysis of data in 
the grounded theory approach sticks closely to this task. 


Link up with Qualitative data, Chapter 14 A 
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Codes, categories and concepts 


The first stage of analysis involves the coding and categorizing of the data. This 
means that the researcher begins to assign bits of the ‘raw data’ to particular 
categories. Careful scrutiny of the data (for example, an interview transcript) 
will allow the researcher to see that certain bits of the data have something in 
common. Possibly they will refer to the same issue. Possibly they will involve 
statements about the same emotion. Possibly they will share the use of a simi- 
lar word or phrase in relation to a specific topic. Whatever it is, the chunks of 
raw data have something in common that the researcher can identify and that 
allows those chunks of data to be coded (tagged) as belonging to a broader 
category. 


Good practice: use a computer to help with the data analysis 


There are a number of commercially available software programs that can help 
researchers with the analysis of qualitative data. It is good practice to store 
the data in a suitable program — such as NVivo — right from the beginning 
of the data collection. Such programs are designed to help researchers to code 
their qualitative data and to develop categories and concepts based on 
these codes. 


Link up with Computer assisted qualitative data analysis, p. 280 4 
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Unlike codes used with quantitative data, in grounded theory, the codes are 
open to change and refinement as research progresses. Initially, the codes will 
be fairly descriptive and are likely to involve labelling chunks of data in terms 
of their content. This is referred to as open coding. As the codes take shape, the 
researcher will look for relationships between the codes — links and associ- 
ations that allow certain codes to be subsumed under broader headings and 
certain codes to be seen as more crucial than others. This is referred to as axial 
coding, since it shifts the analysis towards the identification of key (axial) com- 
ponents. Eventually, the researcher should be in a position to focus attention 
on just the key components, the most significant categories, and concentrate 
his or her efforts on these. This selective coding focuses attention on just the 
core codes, the ones that have emerged from open and axial coding as being 
vital to any explanation of the complex social phenomenon. 

The aim of this process is to arrive at concepts that help to explain the 
phenomenon - basic ideas that encapsulate the way that the categories relate 
to each other in a single notion. These concepts then form the cornerstone for 
the generation of theories that provide an account of things and, in some 
sense or other, explain why things happen as they do. In the words of Strauss 
and Corbin (1990), ‘Concepts are the basis of analysis in grounded theory 
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research. All grounded theory procedures are aimed at identifying, developing, 
and relating concepts.’ 


The constant comparative method 


The grounded theory approach uses the constant comparative method as a 
means of analysing the data. This entails a commitment to comparing and 
contrasting new codes, categories and concepts as they emerge — constantly 
seeking to check them out against existing versions. By comparing each coded 
instance with others that have been similarly coded, by contrasting instances 
with those in different categories, even using hypothetical possibilities, the 
researcher is able to refine and improve the explanatory power of the concepts 
and theories generated from the data. Such comparison helps the researcher to 
refine the codes, categories and concepts by doing the following: 


e highlighting the similarities and differences that exist (promoting better 
categories and descriptions); 

e allowing researchers to integrate categories and codes under common 
headings (facilitating the reduction of complex phenomena to simpler 
elements); 

e allowing researchers to check out their developing theories as they emerge 
(incorporating a way to verify them or refute them at the stage of produc- 
tion rather than after the event). 


By using the constant comparative method the researcher can never lose 
sight of the data, or move the analysis too far away from what is happening 
on the ground. It ensures that any theory developed by the research remains 
closely in touch with its origins in the data — that it remains ‘grounded’ 
in empirical reality. This, of course, is of vital importance to the whole 
approach. 

Note that an important distinction occurs here between the analysis of 
qualitative data and that of quantitative data. Following the grounded theory 
approach, the concepts that form the foundation of the theory/analysis 
emerge from the process of the research activity itself and are not, as is the case 
with quantitative data, established prior to the investigation itself. 


Good practice: data analysis using the constant comparative method 
Analysis should always involve: 


e coding and categorizing the raw data (e.g. interview tapes); 

e constantly comparing the emerging codes and categories with the data; 

e checking them out against new data specifically collected for the purpose; 
e generating concepts and theories that are thoroughly grounded in the data. 
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Completing the research (theoretical saturation) 


How much research is enough? This is, necessarily, rather difficult to specify 
when using the grounded theory approach. Whereas survey researchers using 
quantitative data will know at the start how many items are to be collected, 
and will be able to plan the duration of the research project around this, 
those who opt for a grounded theory approach face the uncomfortable pro- 
spect of not knowing in advance how long the research will take or how much 
data they will need to collect. In principle, they will be expected to continue 
theoretical sampling to test and validate the developing codes, categories and 
concepts until reaching the point of theoretical saturation. 


Key point: theoretical saturation 

Theoretical saturation occurs ‘when additional analysis no longer 
contributes to discovering anything new about a category’ 
(Strauss 1987: 21). 


This point cannot be predicted accurately at the start of the research but the 
researcher will recognize when he or she has arrived there because, at this 
point, the collection of further bits of data ceases to contribute anything 
new to the properties of the codes, categories and concepts being developed. 
Additional field research will tend to confirm the concepts, codes and categor- 
ies that have been developed to date — serving only to confirm and verify the 
theory, not to refine it. Further research, in effect, adds nothing new to the 
researcher’s understanding. 


Theories and grounded research 


The grounded theory approach, when it was originally conceived by Glaser and 
Strauss, was a reaction against the kind of ‘grand’ theories produced through 
the logico-deductive method of science that Glaser and Strauss encountered 
during the late 1950s and early 1960s. As Layder (1993: 42) points out: 


Such theories are generally ‘speculative’ in nature because they have 
not grown directly out of research, and thus remain ungrounded. As 
a consequence, these theories very often lack validity because they do 
not ‘fit’ the real world and thereby remain irrelevant to the people 
concerned. 
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Glaser and Strauss were dissatisfied with this approach and argued that 
a theory developed through the grounded theory approach would have dis- 
tinct advantages over those derived from the more traditional ‘scientific’ 
methods. 

First, it is unlikely that new data will be found that serve to refute the theory. 
Since the theory has been built upon substantive data extracted from field- 
work, the theory should already take account of all the crucial aspects of the 
situation it explains so that there should be no ‘nasty surprises’ waiting to be 
uncovered that will point to a major weakness in the theory. This contrasts, of 
course, with theories that have been logically deduced and that are then sub- 
jected to test by seeing if they work in practice. The ‘scientific’ approach cer- 
tainly is susceptible to such nasty surprises and radical rethinks about the 
theory when findings do not support the theory. In grounded theory, the 
testing is conducted as an integral part of the development of the theory — as 
an ongoing process — not as a separate process conducted after the theory has 
been put together in an abstract form. 

Second, the theory has more credibility. Because the theory emerges from the 
data, there ought to be a comfortable fit between the facts of the situation and 
the theory used to explain them. Put rather simply, the theory is shaped by the 
facts, and therefore there should be a good fit. There should be a close corre- 
spondence between the realities of the situation and the theories used to 
explain them. On this basis, there can be no allegations that researchers are in 
any sense forcing a fit between the data and the theory. By contrast, the process 
of testing theories can involve a search, more or less consciously, for facts that 
fit the theory. Without necessarily intending to do so, researchers can have 
their vision of facts shaped by the theory they are setting out to test. They can 
end up being selective about what they ‘see’ and what they find, managing to 
corroborate their theory by a process of selective perception in which facts that 
do not fit the theory get subtly edited out of the picture as irrelevant or non- 
existent. 


Substantive theory and formal theory 


There are two kinds of theory that can be generated using the grounded theory 
approach. The first is linked closely to the empirical situation that has been the 
subject of study. It is a fairly localized kind of theory and is known as substan- 
tive theory. This is by far the most common kind of theory associated with the 
approach. The other kind of theory is more conceptual, with more general 
coverage and application to circumstances beyond particular settings. This is 
known as formal theory. Ultimately, the aim is to move from a growing body 
of good, grounded substantive theories towards the integration of these in 
higher-level formal theories. 
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Developments in grounded theory 


Since the notion of a grounded theory approach was first conceived, sub- 
sequent works by the originators have moved in slightly different directions. 
This divergence reflects two potential stances that coexist within the original 
grounded theory approach. On the one hand there is Glaser’s version (Glaser 
1978, 1995, 1999, 2003). As Charmaz (2000: 510) argues, ‘Glaser’s position 
often comes close to traditional positivism, with its assumptions of an object- 
ive, external reality [and] a neutral observer who discovers data.’ It rests on the 
belief that the meaning of the data will emerge inductively from the data if 
studied using a suitably neutral methodology. Put simply, the meaning exists 
in the data and the grounded theory approach allows the researcher to extract 
the meaning and develop it into a more abstract theory. By looking closely at 
the data, concepts will occur to the researcher and these concepts can be 
developed and refined by continually comparing them and checking them 
with the original data and with new data collected specifically to verify the 
concepts. 

Contrasting with this, there is Strauss’s version (Strauss 1987; Strauss and 
Corbin 1990, 1998) which moves towards a more post-positivist stance. This 
stance sees the researcher as inevitably imposing some shape and sense on 
to the data. It accepts that researchers cannot be entirely neutral and that 
the meaning attached to any data involves some kind of interpretation by the 
researcher. The meaning will not, cannot, simply emerge inductively from the 
data. In practice, the researcher needs to make sense of the data; that is, to 
interpret the data in a systematic way. To this end, there is extensive guidance 
offered by Strauss and Corbin (1990, 1998) on how researchers should proceed 
with their analysis, turning the spotlight away from the data and more 
towards the procedures for analysing the (qualitative) data. Glaser, for his part, 
has expressed strong reservations about this trend, arguing that there is a 
potential to ‘force’ the data to fit researchers’ categories rather than let the data 
speak for themselves (Glaser 2003). 

Some writers have wanted still greater emphasis on the role of the researcher 
as interpreter rather than discoverer of reality. In line with this, Charmaz 
(2000, 2006) has advocated a constructivist grounded theory, Clarke (2005) has 
adapted grounded theory to postmodernism in what she calls ‘situational analy- 
sis’, and Corbin (Corbin and Strauss 2008) has developed her earlier Straussian 
position to reflect the postmodernist critique with its emphasis on the exist- 
ence of alternative perspectives and constructions of realities. The researcher 
who wants to use the grounded theory approach needs to be aware that such 
debates and developments exist under the umbrella term ‘grounded theory’. 
They might even need to delve into the schisms and controversies in order to 
defend their research as being in line with a particular version of grounded 
theory (see Bryant and Charmaz 2007). Professional researchers and those 
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using grounded theory as a core feature of a PhD would fall into this category. 
Project researchers, however, are less likely to need to pursue the methodology 
to this depth. Provided they adhere to the key points listed below, they should 
be justified in saying that they have adopted a grounded theory approach 
because these items incorporate the core tenets of the approach. 


Good practice: essential ingredients of the grounded theory approach 
The label ‘grounded theory’ should only be used for research that: 


e uses empirical field research as its starting point (the researcher starts the 
fieldwork research early in the investigation); 

e develops its analysis with constant reference to fieldwork data (an iterative 
process); 

e produces explanations that are meaningful for the subjects of research; 

e is geared to modest localized explanations based on the immediate 
evidence; 

e adopts an emergent design (based on theoretical sampling); 

e is linked with qualitative research, exploratory investigations, small-scale 
studies and research focusing on human interaction in specific settings. 


/\ Caution: misuse of the term ‘grounded theory’ 


The term ‘grounded theory’ is often used in a rather loose way to refer 
to approaches that accept some of the basic premises but do not adopt 
the methodological rigour espoused by the originators of the term. All 
too frequently, Glaser and Strauss are cited by way of justification for 
methodologies that actually bear few of the hallmarks of the strict 
conception of ‘grounded theory’. 

Worse than this, there is a very real danger that the term ‘grounded 
theory’ can be hijacked and cited as justification for what is, in reality, 
sloppy research. Taken out of context, ‘grounded theory’ can be mis- 
used as a rationale for approaching research 


without too clear an idea of the topic of research; 

without a clear basis in mind for selecting the sample; 

with a general ‘jump in at the deep end’, ‘try it and see’, ‘find out 
as we go’ approach. 


This is certainly not in tune with the spirit of grounded theory and its 
clear concern with rigour in the analysis of qualitative data. 
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Advantages of the grounded theory approach 


e Suited to small-scale research. From a project researcher’s point of view, use of 
the grounded theory approach can be appealing because it lends itself to 
being conducted by lone researchers working on a limited budget. 

e Recognized rationale for qualitative research. The notion of grounded theory 
has currency in the research community and it has become a recognized 
rationale for qualitative research. Grounded theory provides a standard 
justification that can fend off potential criticism from those who might 
otherwise question the rigour of small-scale qualitative research. 

e Adaptable. The approach is fairly adaptable, lending itself to use with a var- 
iety of qualitative data collection methods (e.g. interviews, observation, 
documents) and forms of data (interview transcripts, fieldwork, texts). 

e Pragmatic. There is a focus on practice (human interaction) and what is 
practical (pragmatic philosophy), which makes the approach well suited 
to studies in areas, such as health, business and education, that are con- 
cerned with understanding and improving matters in terms of inter- 
personal relations. 

e Systematic way of analysing qualitative data. Especially as developed by 
Strauss, this can be helpful to the newcomer who might wonder how on 
earth he or she can make sense of the data and how he or she can move 
towards developing concepts and ultimately theories. 

e Data analysis can draw on computer software for help with the coding and 
sorting of the qualitative data. 

¢ Theory development. The approach includes the means for developing theor- 
etical propositions from data, and should boost project researchers’ con- 
fidence in the realms of theorizing. All researchers — not just the venerable 
experts — are encouraged to arrive at modest theories on the basis of the data 
that they have collected. 

e Explanations are grounded in reality. Concepts and theories are developed 
with constant reference to the empirical data and this means that, unlike 
speculative, abstract theory, they are built on a sound foundation of 
evidence. This ensures that grounded theories are kept in touch with 
reality. 

e Well suited to the exploratory research. The approach permits a degree of flexi- 
bility in both the selection of instances for inclusion in the sample and the 
analysis of the data — both of which are well suited to the exploration of new 
topics and new ideas. 
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Disadvantages of the grounded theory approach 


e Planning. The approach does not lend itself to precise planning. The use of 
‘theoretical sampling’ means that it is not possible to predict in advance the 
nature of the sample that will be used. And the need to achieve ‘theoretical 
saturation’ makes it impossible to state at the outset the eventual size of the 
sample. The voyage of discovery comes with a price and that price is 
the ability to plan all aspects of the research in advance and anticipate when 
the research will be completed. 

e Context. By focusing research on specific instances of behaviour in particular 
settings, there is a tendency to divorce the explanation of the situation 
being studied from broader contextual factors. In particular, there is the 
danger that the theory generated from the data might ignore the influence 
of social, economic and political factors (e.g. power, globalization, migra- 
tion, social class, gender and race inequalities) and the historical back- 
ground to events, which might be vital to any full and valid theory 
explaining the phenomenon. 

e Open-mindedness. An ‘open-minded’ approach to the data is something that 
can operate at a variety of levels. The need to approach things with an ‘open 
mind’ is a fundamental principle of the grounded theory approach but, 
in practice, it raises some awkward questions. Researchers are inevitably 
influenced by prior conceptions based on their own culture and personal 
experience. The question is: how far can these be put to one side for the 
purposes of analysing the data? There is also the controversial point about 
how far previous concepts and theories should be allowed to influence mat- 
ters. Should grounded theory researchers avoid undertaking a literature 
review in order to avoid having their minds ‘contaminated’ by existing 
concepts and theories? And, if so, does this invite the possibility of 
‘reinventing the wheel’ or failing to learn from what has been researched in 
the past? Within the grounded theory approach different researchers adopt 
different positions on such issues. 

e Complexity. The systematic way of analysing data developed by Strauss and 
Corbin can be daunting in terms of the complexity of the process. Indeed, 
Strauss’s development of ‘guidance’ for the analysis of qualitative data actu- 
ally incensed his former collaborator, Barney Glaser, who saw the template 
and framework for analysis as unnecessary and going against the spirit of 
grounded theory by ‘forcing’ categories and codes on to the data, rather 
than letting them naturally ‘emerge’. 

e Positivism. Interpretivists will be unhappy with any suggestion that substan- 
tive theories provide the one correct explanation of things. Yet within the 
writings of Glaser (rather than Strauss) there are occasions when this pos- 
ition is taken. There is a positivistic strand of thought in which the ‘ground- 
ing’ of theory means that it is not liable to be refuted by the later discovery 
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of facts that do not fit the theory. If the theory emerges from the data and is 
meaningful to the participants, then it is a good theory that stands in its 
own right — not open to alternative interpretation. 

Empiricism. The approach can be criticized as being ‘empiricist’. By looking 
to fieldwork data as the source of its theories, and by setting itself against the 
use of general theories, it opens itself to the criticism that it relies too heav- 
ily on the empirical data — expecting an explanation to exist within the 
accumulated data, waiting to be ‘discovered’ by the researcher. This induct- 
ive approach is rather naive about the complex relationship between theory 
and data collection and the way it calls for researchers to approach the data 
without the use of prior theories and concepts (Layder 1998). 
Generalizations. Generalizing from the findings can be misunderstood. It is 
important for those using the grounded theory approach to be clear that 
any generalizations derived from the research are theoretical generalizations. 
They are ‘abstractions’ from the data that are conceptual and which are used 
to construct theories. They are not generalizations in the sense that many 
people might think of them, and grounded theory researchers need to be 
careful to avoid any impression that they are attempting to identify patterns 
of behaviour in the population on the basis of the small, purposively 
selected sample that they have studied. The purpose of grounded theory is 
not to make generalizations from the sample to a wider population as a 
survey approach might do. 
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Checklist for the grounded theory approach 


When undertaking grounded theory research you should feel confident 
about answering ‘yes’ to the following questions: 


Is the aim of the research to generate concepts and 
theory geared to ‘localized’ explanations (substantive theory)? 


Have empirical data been used as the foundation for 
generating the concepts and theories? 


Have measures been taken to adopt an open-minded 
approach to the meaning of the data? 


Have the data been analysed through a systematic 
process of coding and categorization (open coding, etc.)? 


Were the codes, categories and concepts developed 
with constant reference to the fieldwork data (constant 
comparative method)? 


Is there an explicit justification for the choice of the 
initial site for fieldwork in terms of its relevance to the area of 
research? 


Have subsequent sites been selected on the basis of 
theoretical sampling? 


Did the research continue until it reached a point of 
theoretical saturation? 


Has the research focused on issues of practical 
significance to participants in the setting? 


Has the research produced explanations that are 
recognizable to the subjects of research? 


© M. Denscombe, The Good Research Guide. Open University Press. 
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What is the action research approach? e The practical nature of action 
research e Change and professional self-development e Action research 
as a cyclical process e Participation in the research process e Issues 


connected with the use of action research e Advantages of action 
research e Disadvantages of action research e Checklist for action 
research 


What is the action research approach? 


Action research is normally associated with ‘hands-on’, small-scale research 
projects. Its origins can be traced back to the work of social scientists in the late 
1940s on both sides of the Atlantic who advocated closer ties between social 
theory and the solving of immediate social problems. More recently, action 
research has been used in a variety of settings within the social sciences, but its 
growing popularity as a research approach perhaps owes most to its use in 
areas such as education, organizational development, health and social care 
(McNiff and Whitehead 2006; Reason and Bradbury 2006; Somekh 2006; 
Koshy 2010). In these areas it has a particular niche among professionals who 
want to use research to improve their practices. 

Action research, from the start, was involved with practical issues - the kind 
of issues and problems, concerns and needs that arose as a routine part of 
activity ‘in the real world’. This specifically practical orientation has remained 
a defining characteristic of action research. Early on, action research was also 
seen as research specifically geared to changing matters, and this too has 
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remained a core feature of the notion of action research. The thinking here is 
that research should not only be used to gain a better understanding of the 
problems which arise in everyday practice, but actually set out to alter things — 
to do so as part and parcel of the research process rather than tag it on as 
an afterthought which follows the conclusion of the research. This, in fact, 
points towards a third defining characteristic of action research: its commit- 
ment to a process of research in which the application of findings and an 
evaluation of their impact on practice become part of a cycle of research. This 
process, further, has become associated with a trend towards involving those 
affected by the research in the design and implementation of the research — to 
encourage them to participate as collaborators in the research rather than 
being subjects of it. 
Together, these provide the four defining characteristics of action research: 


e Practical nature. It is aimed at dealing with real-world problems and issues, 
typically at work and in organizational settings. 

e Change. Both as a way of dealing with practical problems and as a means 
of discovering more about phenomena, change is regarded as an integral 
part of research. 

e Cyclical process. Research involves a feedback loop in which initial findings 
generate possibilities for change which are then implemented and evalu- 
ated as a prelude to further investigation. 

e Participation. Practitioners are the crucial people in the research process. 
Their participation is active, not passive. 


Action research quite clearly is a strategy for social research rather than a spe- 
cific method. It is concerned with the aims of research and the design of the 
research, but does not specify any constraints when it comes to the means 
for data collection that might be adopted by the action researcher. This point 
is captured by Susman and Evered: 


Action research can use different techniques for data collection . . . Action 
researchers with a background in psychology tend to prefer question- 
naires for such purposes while action researchers with a background in 
applied anthropology, psychoanalysis or socio-technical systems tend to 
prefer direct observation and/or in-depth interviewing... Action 
researchers with any of these backgrounds may also retrieve data from the 
records, memos and reports that the client system routinely produces. 
(1978: 589) 
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The practical nature of action research 


Action research is essentially practical and applied. It is driven by the need to 
solve practical, real-world problems. It operates on the premise, as Kurt Lewin 
put it, that ‘Research that produces nothing but books will not suffice’ (Lewin 
1946: 35). 

But being practical would not be enough to set it apart from other 
approaches to research. After all, many approaches can lay claim to this terri- 
tory. Many can claim with justification to offer applied research. While it 
certainly does embrace this aim, being ‘practical’ has a second sense as far as 
action research is concerned, and it is this which helps to give it a unique 
identity as a research strategy. The research needs to be undertaken as part of 
practice rather than a bolt-on addition to it. 


Good practice: linking research with practice 


Action research [rejects] the concept of a two-stage process in which research 
is carried out first by researchers and then in a separate second stage the 
knowledge generated from the research is applied by practitioners. Instead, 
the two processes of research and action are integrated (Somekh 1995: 34). 


Clearly, if the processes of research and action are integrated, then action 
research must involve ‘the practitioner’ very closely. And this provides a fur- 
ther meaning which can be added to the practical nature of action research: 
practitioner research. However, although it may be linked with the notion of 
practitioner research, it is important to appreciate that action research is not 
exactly the same thing as practitioner research. Edwards and Talbot stress this 
point: ‘Practitioner research can only be designated action research if it is 
carried out by professionals who are engaged in researching ... aspects of 
their own practice as they engage in that practice’ (1994: 52). It is not enough 
for the research to be undertaken as part of the job, because this could include 
all kinds of data gathering and analysis to do with remote people and systems, 
the findings from which might have no bearing on the practitioner’s own 
activity. To accord with the spirit of action research, the researcher needs 
to investigate his or her own practices with a view to altering these in a bene- 
ficial way. 
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Change and professional self-development 


Action research is wedded to the idea that change is good. Initially, this is 
because studying change is seen as a useful way of learning more about the way a 
thing works (Bryman 1989). Change, in this sense, is regarded as a valuable 
enhancer of knowledge in its own right, rather than something that is under- 
taken after the results of the research have been obtained. But, of course, 
the scale and the scope of changes introduced through action research will 
not be grand. The scale of the research is constrained by the need for the 
action researchers to focus on aspects of their own practice as they engage in that 
practice. So change as envisaged by action research is not likely to be a wide- 
scale major alteration to an organization. The element of change will not be in 
the order of a large-scale experiment such as changes in the bonus schemes of 
production workers at Ford’s car assembly plants. No. Because, action research 
tends to be localized and small-scale, it usually focuses on change at the 
micro level. 

One of the most common kinds of change involved in action research is at 
the level of professional self-development. It is in keeping with the notion of 
professional self-development that a person should want to improve practices 
and that this should involve a continual quest for ways in which to change 
practice for the better. Action research provides a way forward for the profes- 
sional which, while it entails a certain degree of reflection, adds the systematic 
and rigorous collection of data to the resources the professional can use to 
achieve the improvement in practice. 

It is important to recognize that reflection may be of itself insufficient to 
make the professional’s endeavour ‘action research’. The reflection needs to be 
systematic if it is to qualify as action research. Merely thinking about your own 
practice — though possibly a valuable basis for improving practice — is not the 
same as researching that action. And here a distinction needs to be made 
between the ‘reflective practitioner’ (Schön 1983) as one who strives for 
professional self-development through a critical consideration of his or her 
practices, and the action researcher who, while also being a reflective prac- 
titioner, adds to this by using research techniques to enhance and systematize 
that reflection. 


Action research as a cyclical process 


The vision of action research as a cyclical process fits in nicely with the 
quest for perpetual development built into the idea of professionalism (see 
Figure 7.1). The purpose of research, though it might be prompted by a specific 
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FIGURE 7.1 The cyclical process in action research 


1 Professional practice 


Instigate Identify problem, or 
change evaluate changes 


= 


Translate findings Systematic and 
into action plan rigorous enquiry 


problem, is seen as part of a broader enterprise in which the aim is to improve 
practice through a rolling programme of research. 

The crucial points about the cycle of enquiry in action research are: (1) that 
research feeds back directly into practice; and (2) that the process is ongoing. The 
critical reflection of the practitioner is not only directed to the identification of 
‘problems’ worthy of investigation with a view to improving practice, but can 
also involve an evaluation of changes just instigated, which can, in their own 
right, prompt further research. It is fair to point out, however, that this is 
something of an ideal and that, in reality, action research often limits itself to 
discrete, one-off pieces of research. 


Participation in the research process 


The participatory nature of action research is probably its most distinctive 
feature, since, in some ways, it goes straight to the heart of conventions associ- 
ated with formal social research. Conventionally, research is the province of 
the expert, the outside authority who is a professional. This researcher more 
often than not initiates the process of research, sets the agenda and designs the 
collection and analysis of data. After the research is concluded, those involved 
might receive some feedback in the form of results from the research. They 
may, or may not, instigate changes on the basis of such findings. Broadly 
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speaking, the act of doing research is separated from the act of making changes 
(Boutilier et al. 1997). Action research, by contrast, insists that practitioners 
must be participants, not just in the sense of taking part in the research but in 
the sense of being a partner in the research. 

Partnerships, of course, take many forms. Grundy and Kemmis (1988: 7) 
argue that ‘in action research, all actors involved in the research process are 
equal participants, and must be involved in every stage of the research’. With 
action research it can be the practitioner, not some outside professional 
researcher, who wants to instigate a change and who consequently initiates 
the research (Elliott 1991). The research, in this sense, is practitioner-driven, 
with the practitioner not just an equal partner but a sponsor and director 
of the research process. And if this sounds radical, there are others who 
would push the matter further, insisting that the practitioner should be the 
dominant partner - calling the shots at all stages and in all aspects of the 
research. 

Even in less radical revisions to the conventional relationship between 
researcher and practitioners, the inclusion of participation as a fundamental 
feature of action research brings with it a shift in the direction of democ- 
ratizing the research process. Control is transferred away from the professional 
researcher and towards the practitioner. Power shifts towards the insider who 
is the practitioner. There is, of course, still a role for the outside expert, but that 
role shifts in the direction of mutual collaboration in the research process, or 
even to the position where the outside expert has the role of facilitator of the 
practitioner’s own project, a resource to be drawn upon as and when the 
practitioner sees fit. 

Behind this shift in the relationship there rests a respect for the practi- 
tioner’s knowledge. Again, this respect may not be unique to action 
research, but it is an aspect of action research which is distinctive and 
built into an approach which is broadly sympathetic to democratizing the 
research process by challenging the separation of expert from lay person 
in research. 

There can be an explicitly political angle to the participatory aspect of 
action research, with its ‘democratization’ of the research process and ‘respect 
for practitioner knowledge’. Zuber-Skerritt (1996) refers to this as ‘emancipa- 
tory’ action research. This type of action research is quite clear about the 
nature of the change which is brought about through action research; it 
is change which challenges the existing system. It is change which goes 
beyond technical matters that can have a bearing on the effectiveness or effi- 
ciency of the professional’s current practice. It is change which goes beyond 
practical matters that can have a bearing on the way practitioners interpret 
the task at hand. For sure, emancipatory action research incorporates these, 
but it also challenges the fundamental framework within which the practice 
occurs. 
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Three approaches to action research 


1 Technical action research aims to improve effectiveness of educational or 
managerial practice. The practitioners are co-opted and depend greatly on 
the researcher as a facilitator. 

2 Practical action research, in addition to effectiveness, aims at the practi- 
tioners’ understanding and professional development. The researcher’s role 
is . . . to encourage practical deliberation and self-reflection on the part of 
the practitioners. 

3 Action research is emancipating when it aims not only at technical and 
practical improvement and the participant’s better understanding, along 
with transformation and change within the existing boundaries and condi- 
tions, but also at changing the system itself of those conditions which 
impede desired improvement in the system/organization. 

(Zuber-Skerritt 1996: 4-5) 


Issues connected with the use of action research 


‘Ownership’ of the research 


The participatory nature of action research brings with it a question mark 
concerning who owns the research and its outcomes. With conventional 
approaches to research this tends to be less complicated. To offer something of 
a caricature, the outsider research initiates the process and approaches practi- 
tioners to gain their permission for the research to be conducted (by the out- 
sider). Having obtained authorization from relevant people, research proceeds, 
with the outsider ‘owning’ the data collected and having full rights over the 
analysis and publication of findings. Of course, complications arise with vari- 
ous forms of sponsored research and consultancy, where authorization might 
include certain restrictions on the rights of the two parties over the research 
and its findings. However, these are likely to be explicitly recognized as a result 
of negotiating access. 

In the case of action research, the partnership nature of work can make 
matters rather less clear-cut. Who is in charge? Who calls the shots? Who 
decides on appropriate action? Who owns the data? These and similar issues 
need to be worked out sensitively and carefully by the partners to ensure 
that there are shared expectations about the nature of participation in action 
research. 
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Ethical issues associated with action research 


The distinct ethical problem for action research is that, although the research 
centres on the activity of the practitioner, it is almost inevitable that the 
activity of colleagues will also come under the microscope at some stage or 
other, as their activity interlinks with that of the practitioner who instigates 
the research. Practitioners are not ‘islands’ — isolated from routine contact with 
colleagues and clients. Their practice and the changes they seek to make can 
hardly be put in place without some knock-on effect for others who operate 
close by in organizational terms. 

The idea that the action researcher is exempt from the need to gain 
authorization, as a consequence, evaporates. Because the activity of action 
research almost inevitably affects others, it is important to have a clear idea 
of when and where the action research necessarily steps outside the bounds 
of collecting information which is purely personal and relating to the practi- 
tioners alone. Where it does so, the usual standards of research ethics must 
be observed: permissions obtained, confidentiality maintained, identities 
protected. 

Two things follow. First, there is a case for arguing that those who engage in 
action research should be open about the research aspect of their practice. It 
should not be hidden or disguised. Second, the need for informed consent 
from those involved in the research should be recognized. 


Good practice: ethics in action research 


Action researchers should respect the rights and sensitivities of colleagues 
and clients who become involved in the research. In practice, as Winter 
(1996: 17) suggests, this means: 


e Permission must be obtained before making observations or examining 
documents produced for other purposes. 

e The researcher must accept responsibility for maintaining confidentiality. 

e The development of the work must remain visible and open to suggestions 
from others. 

e Description of others’ work and points of view must be negotiated with 
those concerned before being published. 


Reflexivity and action research 


Practitioners who engage in action research have a privileged insight into the 
way things operate in their particular ‘work-sites’. They have ‘insider know- 
ledge’. This can be a genuine bonus for research. However, it can also pose 
problems. The outsider — ‘the stranger’ - might be better placed to see the kind 
of thing which, to the insider, is too mundane, too obvious, to register as an 
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important factor. Because the practitioner cannot escape the web of meanings 
that the ‘insider’ knows, he or she is constrained by the web of meanings. The 
outsider ‘expert’ may not have the ‘right’ answer, but can possibly offer an 
alternative perspective which can help the practitioner to gain new insights 
into the nature of the practical problem. 


Se, 
Link up with Reflexivity, pp. 86, 327 e 


So, although action research respects the knowledge of the practitioners, it 
would be rather naive to assume that practitioners’ knowledge - of itself — 
provides all the answers. Particularly in relation to practical action research, 
and emancipatory action research, their aim is to enhance practitioner under- 
standing and this is likely to call upon some modicum of outsider advice. 


Resources and action research 


The action researcher’s investigation is necessarily fairly localized and rela- 
tively small-scale. None the less, the action researcher faces the difficulty of 
trying to combine a probably demanding workload with systematic and rigor- 
ous research. Time constraints, alone, make this hard to accomplish. Even if, as 
should be the case, action research is integrated with practice rather than 
tagged on top of practice, the routine demands of the job are unlikely to be 
reduced by way of compensation. In the short run, prior to positive benefits 
emerging, the action researcher is likely to face extra work. 


Generalizability and action research 


Given the constraints on the scope of action research projects, it might be 
argued that their findings will rarely contribute to broader insights. Located as 
they are in the practitioner’s work-site, there are not very good prospects for 
the representativeness of the data in action research. The setting and constitu- 
ent features are ‘givens’ rather than factors which can be controlled or varied, 
and the research is generally focused on the one site rather than spread across a 
range of examples. Action research, therefore, is vulnerable to the criticism 
that the findings relate to one instance and should not be generalized beyond 
this specific ‘case’. 


Se, 
Link up with Case studies, Chapter 2 o 


In one sense, this reservation needs to be acknowledged. Certainly, practice- 
driven research in local settings hardly lends itself to conclusions with uni- 
versal application. New truths and new theories will be unlikely to find 
foundation in such studies alone. And this caution is worth taking to heart for 
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the action researcher: beware of making grandiose claims on the basis of action 
research projects. However, it can rightly be argued that action research, while 
practice-driven and small-scale, should not lose anything by way of rigour. 
Like any other small-scale research, it can draw on existing theories, apply and 
test research propositions, use suitable methods and, importantly, offer some 
evaluation of existing knowledge (without making unwarranted claims). It is 
the rigour, rather than the size of the project or its purpose, by which the 
research should be judged. 


Advantages of action research 


e Participation. It involves participation in the research for practitioners. This 
can democratize the research process, depending on the nature of the part- 
nership, and generally involves a greater appreciation of, and respect for, 
practitioner knowledge. 

e Professional development. It has personal benefits for the practitioner, as it 
contributes to professional self-development. 

e Practical. It addresses practical problems in a positive way, feeding the 
results of research directly back into practice. In the words of Somekh 
(1995: 340), ‘It directly addresses the knotty problem of the persistent 
failure of research in the social sciences to make a difference in terms of 
bringing about actual improvements in practice.’ 

e Continuous. It should entail a continuous cycle of development and change 
via on-site research in the workplace, which has benefits for the organiza- 
tion to the extent that it is geared to improving practice and resolving 
problems. 


Disadvantages of action research 


e Scope and scale. The necessary involvement of the practitioner limits the 
scope and scale of research. The ‘work-site’ approach affects the represen- 
tativeness of the findings and the extent to which generalizations can be 
made on the basis of the results. 

e Control. The integration of research with practice limits the feasibility of 
exercising controls over factors of relevance to the research. The setting for 
the research generally does not allow for the variables to be manipulated 
or for controls to be put in place, because the research is conducted not 
alongside routine activity but actually as part of that activity. 

e Ownership. Ownership of the research process becomes contestable within 
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the framework of the partnership relationship between practitioner and 
researcher. 

e Work load. Action research tends to involve an extra burden of work for the 
practitioners, particularly at the early stages before any benefits feed back 
into improved effectiveness. 

e Impartiality. The action researcher is unlikely to be detached and impartial 
in his or her approach to the research. In this respect, action research 
stands in marked contrast to positivistic approaches. It is clearly geared to 
resolving problems which confront people in their routine, everyday (work) 
activity, and these people therefore have a vested interest in the findings. 
They cannot be entirely detached or impartial in accord with the classic 
image of science. 
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Checklist for action research 


When undertaking action research you should feel confident about 
answering ‘yes’ to the following questions: 
Does the research project address a practical problem? 


Is there participation by the practitioner in all stages of 
the research project? 


Have the grounds for the partnership between practitioner 
and any outside expert been explicitly negotiated and agreed? 


Is the research part of a continuous cycle of development 
(rather than a one-off project)? 


Is there a clear view of how the research findings will feed 
back directly into practice? 


Is it clear which kind of action research is being used — 
‘technical’, ‘practical’ or ‘emancipatory’? 


Has insider knowledge been acknowledged as having 
disadvantages as well as advantages for the research? 


Is the research sufficiently small-scale to be combined 
with a routine workload? 


Have ethical matters been taken into consideration? 


© M. Denscombe, The Good Research Guide. Open University Press. 


MIXED METHODS 


What is the mixed methods approach? e When is it appropriate to use a 
mixed methods approach? e Types of mixed methods design e The link 
between the methods e Mixed model research designs e Pragmatism 


and the mixed methods approach e Advantages of the mixed methods 
approach e Disadvantages of the mixed methods approach e Checklist 
for mixed methods research 


The term ‘mixed methods’ applies to research that combines alternative 
approaches within a single research project. It refers to a research strategy that 
crosses the boundaries of conventional paradigms of research by deliberately 
combining methods drawn from different traditions with different underlying 
assumptions. At its simplest, a mixed methods strategy is one that uses both 
qualitative and quantitative methods. 

There is nothing startlingly new about the strategy. It is easy to find examples 
throughout the history of social research in which researchers successfully 
mixed their methods. However, the identification of something called the 
‘mixed methods approach’ is relatively new. As an approach, with a recognized 
name and research credibility, it has only come to the fore in recent years 
championed by writers such as Creswell (2009), Creswell and Plano Clark 
(2007), Greene (2007), Johnson and Onwuegbuzie (2004) and Tashakkori and 
Teddlie (1998, 2003, 2009). This chapter focuses mainly on this new ‘move- 
ment’. Partly this reflects its new rise to prominence; partly it reflects the valu- 
able way this movement has explored the underlying rationale for mixing 
methods. It would be wrong, however, to infer that others have not explored 
the prospects of combining qualitative and quantitative research and made 
valuable contributions in their own right to a broader notion of a mixed 
methods strategy in social research (see, for example, Silverman 1985, 2010; 
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Bryman 1988, 2006; Gorard and Taylor 2004; Layder 1993, 1998). While such 
writers might not accept all the tenets of the ‘new mixed methods approach’ 
(in particular the use of pragmatism as its philosophical foundation), there is 
consensus on one fundamental point — the belief that treating qualitative and 
quantitative approaches to research as incompatible opposites is neither helpful nor 
realistic when it comes to research activity. 

A number of different names have been given to mixed methods strategies. 
‘Mixed methodology’, ‘multi-strategy research’, ‘integrated methods’, ‘multi- 
method research’ and ‘combined methods’ research are just some of the alter- 
natives. The names bear testimony to the variety of ways in which research can 
be mixed and the many aspects of the research process that can be involved. 
The most common form of mix, however, involves a mix of methods and, in 
practice, the term mixed methods has come to be used in a general sense to 
cover the spectrum of ways in which mixing can occur in the research process. 
In effect, it has given its name to a broad approach to research. 


What is the mixed methods approach? 


The mixed methods approach has three characteristic features that set it apart 
from other strategies for social research. These can be summarized as follows: 


1 Use of qualitative and quantitative approaches within a single research project. At 
the heart of the approach is the idea that researchers can bring together 
within a single research project certain elements that have conventionally 
been treated as an ‘either/or’ option. In most cases, the distinction is drawn 
between ‘qualitative’ and ‘quantitative’ with researchers variously writing 
about using both qualitative and quantitative methods, qualitative and 
quantitative data or qualitative and quantitative research. 

2 Explicit focus on the link between approaches (triangulation). The mixed 
methods approach emphasizes the need to explain why the alternative 
approaches are beneficial and how the alternatives are to be brought 
together. Particular attention is given to the design of mixed methods 
research and especially the role of triangulation in justifying the use of 
the alternative approaches. 


Link up with Triangulation, Appendix 3 A 


3 Emphasis on practical approaches to research problems (pragmatism). The mixed 
methods approach is ‘problem-driven’ in the sense that it treats the research 
problem - more specifically answers to the research problem - as the overrid- 
ing concern. Other approaches, of course, share a concern for practical 
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solutions to real-world problems. But, advocates of the mixed methods 
approach regard it as the crucial driving force behind decisions about which 
methods to use. This means they are prepared to use methods that come 
from different philosophical traditions, provided their use produces find- 
ings that are of practical value for addressing the research problem. The 
mixed methods approach, in this sense, challenges the premise that social 
research should strive for consistency between the various components 
of the research process — the questions, the design, the data collection, 
the data analysis - in terms of their underlying assumptions about the 
nature of the social world and the possibilities this opens for research. 
It adopts instead a pragmatist position that allows it to bring together 
methods drawn from ‘paradigms’ of research conventionally regarded as 
incompatible. 


When is it appropriate to use a mixed methods approach? 


Drawing on reviews of published mixed methods studies (Greene et al. 1989; 
Rocco et al. 2003; Bryman 2006; Collins et al. 2006), it is evident that social 
researchers use mixed methods strategies for one or more of the following 
purposes. 


Improved accuracy 


Researchers can improve their confidence in the accuracy of findings through 
the use of different methods to investigate the same subject. In line with the 
principles of triangulation, the mixed methods approach provides the 
researcher with the opportunity to check the findings from one method 
against the findings from a different method. In the words of Greene et al. 
(1989: 259) this use of the mixed methods approach ‘seeks convergence, cor- 
roboration, correspondence of results from the different methods’. Where 
different methods produce data that are more or less the same, the researcher 
can feel more confident in assuming that the findings are accurate. 

Using the same principle of triangulation, researchers can also check on the 
impact of a particular research method. For example, if a researcher wants 
to know whether online questionnaires ‘distort’ the kind of answers that are 
given by respondents, he/she could compare the findings from this method of 
data collection with findings from a similar set of respondents completing a 
conventional paper questionnaire. If there is no marked difference between the 
methods in terms of the data collected, the researcher can be more confident 
that the use of the online questionnaires does not have a distorting impact 
on the data (at least, no impact that is different from the conventional paper 
questionnaire). 
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The mixed methods approach is valuable, too, as a means of developing 
research instruments. For example, researchers who are designing a question- 
naire for use in a survey can employ qualitative data through focus groups and 
interviews to improve the validity of a subsequent survey questionnaire that 
produces quantitative data. The qualitative data collected through interviews 
and focus groups can be valuable initially as a way of shaping the kind of 
questions that are relevant and then subsequently to ensure that the questions 
have been worded suitably. 

In sum, then, the mixing of methods can be a valuable research strategy for: 


e the validation of findings in terms of their accuracy; 
e checking for bias in research methods; 
e the development of research instruments. 


/\ Caution: non-corroboration 


When the mixed methods approach is being used to check the validity 
of findings, any disparity between the findings can pose a challenging 
and possibly unwelcome occurrence. The mixed methods approach, to 
a large degree, operates on the assumption that findings will coincide 
and that this will be a positive contribution to the research project. If 
the findings do not coincide, however, the researcher can be faced 
with a problem. /n principle, the non-alignment of results need not be 
a setback. It should spur further research to investigate the difference 
in findings. The contradictions could be very interesting — revealing 
limitations to the methods of data collection or teasing out fascinating 
complications within the phenomenon being studied. In practice, 
though, non-corroboration can have negative consequences. Project 
researchers and contract researchers can be under pressure to arrive 
at conclusions within a given time-frame. They can be expected to 
produce reports that have to be delivered by a given deadline. But, at 
the end of the project they could be left with contradictory findings 
from the alternative methods (or data sources) and no firm conclusion. 
Where people look to research to provide concrete findings or practical 
solutions to a pressing problem, they might not be very receptive to 
conclusions that are based on conflicting data from different methods. 
The audience for the research (e.g. funders, examiners) might feel 
dissatisfied that the completion of the project does not provide a clear- 
cut finding. They might even interpret things to mean that one or other 
of the methods (or data sources) employed was ‘wrong’ and should 
not have been used within the project. 


MIXED METHODS 141 


A more complete picture 


The use of more than one method can enhance the findings of research by 
providing a fuller and more complete picture of the thing that is being studied. 
The benefit of the mixed methods approach in this instance is that the data 
produced by the different methods can be complementary. They can provide 
alternative perspectives that, when combined, go further towards an all- 
embracing vision of the subject than could be produced using a mono-method 
approach. In terms of the mixed methods approach this tends to involve the 
use of both qualitative and quantitative methods as a means of seeing things 
from alternative perspectives and, thereby, getting a more complete overview 
of the subject. 

Consider, for example, research that focuses on smoking and the health- 
related behaviour of young people. A quantitative survey approach would 
certainly produce evidence of the extent of smoking by under-16s and could 
provide good demographic details that will help us to understand the social 
backgrounds of those involved. A series of focus groups involving young 
smokers could be used to provide qualitative data that would help to explain 
why the underage smokers started smoking and what motivates them to con- 
tinue. In their own right, both the quantitative and the qualitative methods 
are valuable and can make a worthwhile contribution to what is known on 
the matter. Combining the methods, however, allows the researcher to pro- 
duce a fuller account of the situation that covers not only the scale of the issue 
(e.g. numbers involved, age, sex, ethnic group) but also gives some insight into 
the motivational factors that give rise to the behaviour (e.g. self-image, stress, 
peer groups). 


Compensating strengths and weaknesses 


Recognizing that different methods have their respective strengths and weak- 
nesses, one of the valuable uses of mixing methods is to offset any inherent 
weakness or bias in a particular method by combining it with a different one 
that can compensate for this weakness or bias. In Denzin’s words: ‘By combin- 
ing multiple observers, theories, methods and data sources, [researchers] can 
hope to overcome the intrinsic bias that comes from single-methods, single 
observer, and single theory studies’ (1989: 307). 

A shrewd combination of methods allows the researcher to exploit the 
strengths of a particular method without leaving himself/herself vulnerable to 
criticism in connection with that method’s weakness. To illustrate the point, 
a researcher who chooses to use semi-structured interviews as the main data 
collection method might also choose to supplement this method with the 
use of a closed-answer questionnaire. The choice of semi-structured interviews 
could reflect the researcher’s need to gain an in-depth understanding of the 
motivations of a particular group of people — say, airline pilots. The choice 
of semi-structured interviews might seem appropriate because this method is 
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particularly good at allowing the researcher to explore in depth the thoughts, 
feelings and reasoning of the pilots. But because interviews generally take 
time to organize, conduct and analyse, the numbers involved are generally 
quite small - which makes the researcher vulnerable to the criticism that 
the data are not representative. To compensate for this weakness, a mixed 
methods strategy can be adopted that involves a questionnaire survey of air- 
line pilots, possibly including pilots from other countries and employed by 
other airlines. The initial findings from the interviews can be checked with 
the broader population of airline pilots using questionnaires which, as a 
research method, lend themselves far better to use with large numbers 
across wide geographical areas. Questionnaires, of course, would not have 
provided the kind of depth of information that the semi-structured interviews 
did but, by combining the methods, the researcher is in a position to avoid 
potential criticism linked to either the relatively small sample size associated 
with the interview method or the relative superficiality of data collected via 
questionnaires. 


Developing the analysis 


Within a mixed methods strategy, contrasting methods can be used as a means 
of moving the analysis forward, with one method being used to inform 
another. In this sense, an alternative method is introduced as a way of building 
on what has been learned already through the use of the initial method. It is 
used to produce further data that might shed light on things. This is different 
from using another method to produce a fuller picture of things; the new 
method is introduced specifically to address a research issue arising through 
findings produced by another method. 

A classic study illustrates this use of amixed methods strategy: the Hawthorne 
studies. A series of studies was conducted at the Western Electric company’s 
Hawthorne plant in Chicago between 1924 and 1933. The aim of the studies 
was to discover which factors affected worker productivity. Initially, manage- 
ment at the plant conducted a sequence of quasi-experiments in which they 
altered the level of lighting to see if this was linked with productivity. 
Subsequently, researchers from Harvard Business School joined the investiga- 
tion and conducted a further series of studies designed to investigate the 
link between fatigue, monotony and worker productivity (Mayo [1933] 
2003; Roethlisberger and Dickson [1939] 2003). All these controlled, quasi- 
experimental studies produced rather puzzling and inconclusive results and 
this caused the researchers to introduce a new method of enquiry in 1932. 
Instead of trying to observe the factors that affected productivity they decided 
to ask the workers what motivated them. The researchers embarked on a huge 
programme of interviews with the workforce. What they found out about 
workers’ motivation through the interviews helped the researchers to make 
sense of the findings from the experimental/observational phases of the 
research and it laid the foundations for a further observational phase of the 
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research designed to check their conclusions. The interviews revealed that 
the workers were motivated by factors other than just money; there were per- 
sonal and social rewards from work that affected their productivity as well. 
Once the researchers understood this, they could see that the results of 
the earlier observation studies had been influenced by the eponymous 
‘Hawthorne effect’ in which the process of observing people can have an 
impact on the nature of those people’s behaviour. In the case of the 
Hawthorne studies, members of the experimental groups reacted to being 
observed in as much as they felt more valued and appreciated by management. 
This increased the group’s morale which had the knock-on effect of increasing 
their productivity. This impact on their productivity had nothing directly to 
do with the factor of fatigue that was being investigated and it helped to 
explain why changes introduced to affect the level of worker fatigue did not 
have a predicted impact on productivity. A confounding variable — worker 
morale — was interfering with the process. But the crucial point in relation to 
mixing methods is that by interviewing the workforce the researchers were 
able to understand why their quantitative methods had not supplied ‘the 
answer’. The qualitative research, in this sense, informed the quantitative 
research. It explained that worker productivity owed as much to worker 
motivation as it did to physical fatigue, and that worker motivation stemmed 
from social and psychological factors such as morale, recognition and group 
cohesion. 


9o 
Link up with Experiments, Chapter 3 e 


An aid to sampling 


Using a mixed methods strategy, information from one method can be used as 
the basis for selecting a sample of people who will participate in the research 
through a different and contrasting method. Sometimes this involves the 
use of a quantitative pilot study to provide background information that 
researchers use to guide their selection of the individuals who will participate 
in subsequent qualitative research involving things like focus groups and 
interviews. Other times, as Collins et al. (2006) found, qualitative research 
can precede quantitative research as a means of identifying the appropriate 
kind of people to take part in a piece of research and as a means of filtering 
out those who might not be suitable. Indeed, interviews and focus groups can 
be used in a variety of ways to help recruit participants for a survey or panel 
study. The qualitative methods can aid the quantitative methods by identify- 
ing the kind of people who are required for the study, by finding out what 
things might influence their willingness to take part in the study, and even 
by finding out at a later stage what caused some recruits to drop out from 
the research. 
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Good practice: being clear about the aims 


Researchers need to be clear about their reason for mixing methods and 
should specify whether it is being done because of its benefits for: 


e improved accuracy; 

e amore complete picture; 

e compensating strengths and weaknesses; 
e developing the analysis; 

e sampling. 


Types of mixed methods design 


Researchers who advocate the use of the mixed methods approach generally 
make a distinction between QUAL and QUAN where: 


e QUAL stands for qualitative data in the form of text or pictures that provide 
the basis for interpretations of the meaning they convey; 

e QUAN stands for quantitative data in the form of numbers that provide 
objective measurements of observed events. 


The mix of QUAL and QUAN can take a number of forms and it is possible 
to construct a complex range of possibilities for combining them within a 
single research project. Basically, though, the key decisions facing the social 
researcher in this context boil down to two things: 


e In what order should the alternative methods be used? 
e Should greater importance be attached to one or other of the alternative 
methods? 


When a research design involves the use of methods or strategies drawn from 
different paradigms, the order in which the methods/strategies are used is 
significant. Given that they are used as part of the same investigation of the 
same research question, the sequencing reflects the researcher’s beliefs about 
how the combination of methods and strategies works best. For some purposes 
it might be better to have QUAN followed by QUAL; for other purposes 
the opposite might be true. Possibly, more phases within a research project 
might be most useful, in this case calling for a sequence of three or more 
QUAL/QUAN components. 

There might be occasions when the research is best served by conducting 
the contrasting alternatives alongside each other at the same time. This would 
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involve the ‘simultaneous’ use of QUAL and QUAN. A further variation might 
involve the use of QUAL and QUAN with different ‘levels’ of research. Such a 
‘multilevel research’ design, for example, might use a QUAL approach to the 
study of individual pupils in classroom (e.g. their feelings and experiences in 
relation to homework), use a QUAN approach to investigate features of whole 
classes (e.g. test scores and attendance rates), use a QUAN approach for looking 
at the school (e.g. truancy rates and exam pass rates), and complement these 
with a QUAL approach looking at the history and environment of the school 
(e.g. reports by school inspectors, local newspaper articles). Of course, a multi- 
level design such as this could operate either simultaneously or sequentially. 

Taking the factors of sequencing and level into account, a large number of 
possibilities exist in relation to mixed methods research design. The basic 
principles of such designs, however, can be depicted in Table 8.1. 


Table 8.1 Mixed methods research design possibilities 


QUAL > QUAN 
QUAN —> QUAL 
QUAN > QUAL — QUAN 


Sequential studies e.g. 


Simultaneous studies e.g. QUAN } 

QUAL 
Multilevel designs e.g. QUAN at macro level | 
(simultaneous or sequential) QUAL at micro level 


Research designs that incorporate aspects of both QUAL and QUAN need 
not necessarily attach equal weight to the two. Indeed, there is a strong likeli- 
hood that researchers will tend to regard one as the ‘main’ and the other as the 
‘subsidiary’ counterbalance or check. For example, the researchers might start 
by collecting and analysing a relatively small amount of qualitative data as 
an exploratory phase of the investigation. They might use interviews or focus 
groups to get a feel for the relevant issues and to explore the way things are 
perceived. Armed with this knowledge, the researchers can then produce a 
questionnaire (or other quantitative research instrument) that can build on 
information obtained in advance. Some psychology and marketing research 
adopts this approach, using the QUAL as a preliminary checking device to 
ensure the validity of a QUAN research instrument. In this approach, it is 
the QUAN data that are treated as the most important material for the 
investigation. 

In contrast to this, the design might treat QUAL as the dominant, perhaps 
preceding the QUAN subsidiary. An example of this would be the use of a 
detailed case study that is followed up with small-scale questionnaire survey to 
check the case study findings. The research design, in this instance, centres on 
the collection and analysis of qualitative data. It might, for example, consist of 
interview data collected from employees within a particular business. From 
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these interviews the researchers might reach certain conclusions. At this point, 
they might choose to conduct a survey of other businesses to check if these 
conclusions tend to be borne out in other situations. The key points might 
be included in a questionnaire distributed to a representative sample of busi- 
nesses nationwide, with the hope that the responses they receive will cor- 
roborate the conclusions based on the detailed case study. The main thrust of 
the investigation, in this example, is the QUAL (interviews) with the QUAN 
(questionnaire survey) being used in a supporting role. 

In a simplified depiction of the range of possibilities, the sequencing and 
dominance of methods from different research paradigms might take one of 
the following forms (Table 8.2). 


Table 8.2 Sequencing and dominance in mixed methods research designs 


Equivalent status e.g. QUAL —> QUAN 

QUAN —> QUAL > QUAN 
Dominant/less dominant status e.g. qual > QUAN 

QUAL — quan 


/\ Caution: the distinction between QUAL and QUAN 


Although the labels QUAL and QUAN are convenient to use and easy to 
understand, a clear dividing line between qualitative and quantitative 
approaches can be hard to sustain (Reichardt and Rallis 1994; 
Hammersley 1996; Halfpenny 1997; Coxon 2005). In reality, it is 
hard to find examples of methods that are entirely QUAL or entirely 
QUAN. As Gorard and Taylor (2004: 6) make the point, ‘All methods of 
social science research deal with qualities, even when the observed 
qualities are counted. Similarly, all methods of analysis use some 
form of number, such as “tend, most, some, all, none, few” and so 
on.’ So, qualitative social research involves some quantification and 
measurement; quantitative social research involves some element of 
interpretation. The labels QUAL and QUAN, then, really refer to the 
amount of emphasis specific methods put on interpretation (QUAL) 
or measurement (QUAN). 


MIXED METHODS 147 


The link between the methods 


The mixed methods approach does not mean ‘anything goes’. As with cook- 
ing, there are certain ingredients that go together well and others that do not. 
Ingredients need to be mixed in the right order and in the right amount if they 
are to be successful, and there is no sense in which the mixed methods 
approach promotes a careless throwing together of ingredients in a haphazard 
fashion: that would be a recipe for disaster. To qualify as a mixed methods 
strategy, the research needs to have a clear and explicit rationale for using the con- 
trasting methods. Simply combining different methods is not enough; there 
also needs to be a clear vision of why the different methods, when combined, 
will be of benefit for the research. There needs to be an explanation of how the 
various methods are interlinked and how they feed into one another. Are the 
findings from one phase of the investigation being used to direct the investiga- 
tion at a subsequent stage? Are the findings from one kind of method being 
compared with those from another kind of method at the stage of analysis? 
Increasingly mixed methods research is being judged according to how clearly 
each part links with the others and what precise benefits stem from such links. 


Good practice: the mix of methods 
The mix of methods should be explained in terms of: 


e sequence: the order in which the different methods are used; 

e priority: the emphasis place on the alternative methods; 

e relationship: how the various methods interlink and how they ‘inform’ 
one another. 


Mixed model research designs 


The mix of alternative approaches within a single research project can involve 
more than just the data collection methods or the kind of analysis. It is pos- 
sible to mix methods at a more fundamental level, building the mix into the 
fabric of the research strategy and design so that it is not just the methods 
and analysis that are mixed, but the nature of the research questions and 
whole style of research. This is termed ‘mixed model’ research (see Tashakkori 
and Teddlie 1998: 57). Conventionally, social research has followed one of 
two broad paths when it comes to the style and nature of the investigation. 
Either the research has set out to test hypotheses and check predictions based 
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on theories by conducting ‘confirmatory investigations’. Or, it has aimed to dis- 
cover things by engaging in ‘exploratory investigations’ which are not based on 
any firm hypothesis at the beginning. Mixed model research designs break the 
mould by incorporating both kinds of approach within the same project. 


/N Caution: mixed methods and grounded theory 


Mixed methods research designs that are based on the sequencing of 
the QUAL and QUAN components of research do not sit comfortably 
with the kind of emergent research designs associated with grounded 
theory approaches or some other forms of qualitative enquiry. Within 
the mixed methods approach, discussions of the research design tend 
to presume that the sequencing of the QUAL/QUAN components is 
something that can and should be determined at the outset of the 
research. For many purposes, this is good practice. However, it does 
not really provide for the kind of flexibility required by emergent 
research designs. Nor does it really allow for ‘developing the analysis’ 
in which unexpected results can prompt the researcher to bring on 
board a different research method in direct response to unforeseen 
outcomes. 


Pragmatism and the mixed methods approach 


Pragmatism is generally regarded as the philosophical partner of the mixed 
methods approach. It provides a set of assumptions about knowledge and 
enquiry that (1) underpins the mixed methods approach; and (2) distinguishes 
the approach from purely quantitative approaches that are based on a phil- 
osophy of positivism and purely qualitative approaches that are based on a 
philosophy of interpretivism. In the context of research, pragmatism tends 
to revolve around the following core ideas: 


e Knowledge is based on practical outcomes and ‘what works’. The key cri- 
terion for judging knowledge is how useful it is perceived to be and how 
well it works when applied to some practical problem. 

e Research should test what works through empirical enquiry. 

e Thereis no single, best ‘scientific’ method that can lead the way to indisput- 
able knowledge. 

e Knowledge is provisional. What we understand as truth today may not be 
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seen as such in the future. Knowledge is seen as a product of our times. It 
can never be absolute or perfect because it is inevitably a product of the 
historical era and the cultural context within which it is produced. The quest 
for absolute ‘Truth’ is consequently seen as a hopeless cause. 

e Traditional dualisms in the field of philosophy and science are regarded 
as not helpful. In particular, there is scepticism about the distinction 
between quantitative and qualitative research, but there is also a rejection 
of distinctions like facts/values, objectivism/subjectivism and rationalism/ 
empiricism. 


For some mixed methods researchers, these tenets of pragmatism leave the 
door open for the use of purely quantitative research or purely qualitative 
research — providing the use of either in isolation can produce the kind of 
findings that work sufficiently well to answer a research problem. It leads them 
to regard the use of mixed methods as a third alternative - another option 
open to social researchers if they decide that neither quantitative nor qualita- 
tive research alone will provide adequate findings for the particular piece of 
research they have in mind. Rather than see positivism and interpretivism 
as having become obsolete, these researchers regard the mixed methods 
approach as simply an alternative research strategy that can be used if, and 
when, necessary. ‘What answers my question’ is their guiding principle. 

Others, seeking to challenge what are regarded as sterile and unproductive 
dualisms, favour a search for common ground between the ‘old’ philosophies 
of research. They focus on areas of compatibility between quantitative and 
qualitative research, and between positivism and interpretivism. Theirs is an 
eclectic approach in which the traditional paradigms of research are brought 
together and emphasis is placed on how elements of the paradigms can be 
combined to good effect. ‘What meets my needs’ is their guiding principle. 

There are some mixed methods researchers, as we have seen, who operate on 
the assumption that good social research will almost inevitably require the use 
of both quantitative and qualitative research in order to provide an adequate 
answer. For them, the mixed methods approach is built on the belief that not 
only is it allowable to mix methods from alternative and incompatible para- 
digms of research but it is also desirable to do so in order to provide answers 
that work - or, at least, that work better than those based on the use of just 
quantitative or just qualitative research. For some of the main advocates of the 
mixed methods approach (e.g. Tashakkori and Teddlie 1998, 2003; Johnson 
and Onwuegbuzie 2004; Teddlie and Tashakkori 2009) pragmatism represents 
an advance in thinking about social research — a new paradigm that is replacing 
the earlier paradigms based on positivism and interpretivism. ‘What works 
best’ is their guiding principle. 

Despite this difference of emphasis within the mixed methods camp, there 
is still some broad agreement that social research should not be judged by 
how well it fits in with the quantitative paradigm (positivism) or with the 
premises of the competing camp — the qualitative paradigm (interpretivism). 
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The decision to use a mixed methods strategy should be based on how useful 
the methods are for addressing a particular question, issue or problem that is 
being investigated. For those who adopt the mixed methods approach, the 
crucial consideration is how well the research tools work rather than how well 
they fit within a specific research philosophy. 


$% 


Key issues in the mixed methods approach 


e Order: in what order do the methods get used? 

e Timing: at what points are methods changed or combined? 

e Proportion and priority: which methods are seen as dominant? 

e Comparison and contrast: linked with triangulation, are the methods 
similar or markedly different? How do the various methods inform one 
another? 

e Use and benefit: for what purposes are the alternative approaches 
combined? 


Advantages of the mixed methods approach 


e A more comprehensive account of the thing being researched. A mixed method 
approach can provide a fuller description and/or more complete explan- 
ation of the phenomenon being studied by providing more than one 
perspective on it. By encouraging the use of qualitative and quantitative 
methods and by facilitating a blend of exploratory and explanatory research, 
the findings are likely to address a wider range of the questions relating to 
‘how’, ‘why’, ‘what’, ‘who’, ‘when’ and ‘how many’. 

e Clearer links between different methods and the different kinds of data. The 
mixed methods approach places emphasis on the integration of alternative 
approaches and encourages the researcher to provide an explicit account of 
how and why the different methods and data complement each other. 
The approach is particularly sensitive to the nature of the mix and how 
the qualitative methods relate to the quantitative methods in the research. 
To this extent, the mixed methods approach avoids any arbitrary ‘mix- 
and-match’ approach where methods get thrown together in a random or 
ad hoc way. 

e Triangulation. Following on from its emphasis on the rationale for combin- 
ing different approaches, the mixed methods approach involves a height- 
ened sensitivity to the nature of triangulation. In effect, the use of a mixed 
methods approach calls for a clear appreciation of ‘triangulation’ and makes 
good use of its potential. 

e A practical, problem-driven approach to research. As a movement, the mixed 
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methods approach is problem-driven rather than theory-driven. Its under- 
lying philosophy is that of pragmatism. As such, it gives voice to the general 
practice of a large proportion of social researchers who have, for years, 
used methods from different research paradigms because it has been prac- 
tical to do so. 


Disadvantages of the mixed methods approach 


e The time and costs of the research project can increase. The combination of 
phases can extend the time-frame for research design and data collection. 

¢ The researcher needs to develop skills with more than one method. The researcher 
needs to develop and exercise skills covering both qualitative and quantita- 
tive approaches. This adds demands on the researcher and opens up the 
possibility of ‘missing the mark’ for experts in either camp. 

e The QUAL/QUAN distinction tends to oversimplify matters. When using the 
QUAL/QUAN distinction, therefore, mixed methods researchers need to be 
aware that the clarity and simplicity of the terms mask a more complicated 
reality. 

e The mixed methods designs that are generally recommended do not allow for 
emergent research designs. There is, though, no reason in principle why a 
mixed methods approach cannot adopt an emergent design. 

e The underlying philosophy of the mixed methods approach — pragmatism — is open 
to misinterpretation. There is a common-sense use of the word pragmatic 
which implies expediency and a certain lack of principles underlying a 
course of action. There is the danger, then, that the mixed methods approach 
gets associated with this understanding of the word and thus becomes 
regarded as an approach in which ‘anything goes’. It should be stressed that 
this is not the philosophical meaning of pragmatism and it is not a meaning 
that should be associated with the mixed methods approach. 

e Findings from different methods might not corroborate one another. If findings 
from the different methods do not corroborate one another, the researcher 
can be faced with the need to extend the research to unravel the reasons 
for this. 
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Checklist for mixed methods research 


When undertaking mixed methods research you should feel confident about 
answering ‘yes’ to the following questions: v 


Does the research do one or more of the following: 


make use of more than one research method? 
combine more than one kind of research data? 
ask questions based on different models 
(exploratory/explanatory)? 


Does the research have a clear design showing the 
sequence and dominance of the QUAL and QUAN parts? 


Are the QUAL and QUAN components integrated in an 
explicit way? 
Does this occur in relation to: 


e the methods of data collection? 
e the qualitative and quantitative data? 
e the kinds of data analysis? 


Has a clear statement been made about the benefits of 
using a mixed methods strategy compared with using just one 
method? 


Is the rationale for using a mixed methods strategy based 
on one or more of the following criteria? 


the needs of the particular research question; 
improving the accuracy of data; 

providing a more complete picture; 
developing the analysis; 

an aid to the sampling of participants; 

a means of compensating the strengths and 
weaknesses of particular methods; 

the principle that mixed methods research is 
fundamentally good practice. 


© M. Denscombe, The Good Research Guide. Open University Press. 


METHODS OF SOCIAL 
RESEARCH 


There are four main methods that social researchers can use: questionnaires, 
interviews, observation and documents. Each method provides a tool for the 
collection of empirical data. In many ways they are the social scientist’s 
equivalent of the microscope, the scales and the micrometer that natural sci- 
entists might use in relation to material objects in the physical world. They are 
tools that help the researcher to gain: 


e aclearer picture of things; 
e an accurate measurement of things; 
e facts and evidence about the subject matter. 


Selecting methods — a matter of ‘horses for courses’ 


In the following chapters there is guidance about the key features of each 
method. However, before choosing and using any of the tools, it is important 
to consider five points. 

First, although certain research strategies have come to be associated with 
the use of certain research methods this does not rule out the possibility 
of choice. Surveys tend to be linked with questionnaires, for example, while 
ethnography tends to be associated with participant observation. And there 
are some sound theoretical reasons which explain the tendency of strategies to 
be linked with particular methods. The linkage, however, is not watertight and 
it is important for the project researcher to recognize that there is likely to be 
some scope to choose among different methods for use within any specific 
strategy. 
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Second, each of the methods has its particular strengths and weaknesses. 
Each method approaches the collection of data with a certain set of assump- 
tions and produces a type of data whose usefulness depends on what the 
researcher is trying to achieve. This means that none of the possible methods 
for data collection can be regarded as perfect and none can be regarded as 
utterly useless. 

Third, when it comes to choosing a method, researchers should base their 
decision on the criterion of ‘usefulness’. Rather than trying to look for a 
method that is superior to all others in any absolute sense, they should look for 
the most appropriate method in practice. Researchers should ask themselves 
which method is best suited to the task at hand and operate on the premise 
that, when choosing a method for the collection of data, it is a matter of ‘horses for 
courses’. 

Fourth, research methods do not need to be seen as mutually exclusive. If no 
method is intrinsically better than others, and if methods are to be chosen 
pragmatically, then this opens up the possibility of combining methods and 
using a mixed method approach (see Chapter 8). More than one method can be 
used for the research with the clear intention that weaknesses in one method 
can be compensated for by strengths in another method. 

Fifth, the use of more than one method allows the researcher to use triangu- 
lation. The combination of different methods can be used by the researcher 
to look at the research topic from a variety of perspectives, as a means of 
comparison and contrast, with each method approaching the topic from a 
different angle. 


Link up with Triangulation, Appendix 3 - 
"ee 


QUESTIONNAIRES 


What is a questionnaire? e When is it appropriate to use a 
questionnaire? e What kinds of data are collected by questionnaires? 
e Planning the use of questionnaires e Internet questionnaires 

e Vital elements of a research questionnaire e The length of the 


questionnaire e Devising the questions e Types of questions 
e Evaluating questionnaires e Advantages of questionnaires 
e Disadvantages of questionnaires e Checklist for the use of 
questionnaires 


What is a questionnaire? 


There are many types of questionnaires. They can vary enormously in terms 
of their purpose, their size and their appearance. To qualify as a research ques- 
tionnaire, however, they should do the following: 


e Be designed to collect information which can be used subsequently as data for 
analysis. As a research tool, questionnaires do not set out to change people’s 
attitudes or provide them with information. Though questionnaires are 
sometimes used for this purpose — for instance, as a way of marketing a 
product — it is not strictly in keeping with the spirit of a research question- 
naire, whose purpose is to discover things. 

e Consist of a written list of questions. The important point here is that each 
person who answers the particular questionnaire reads an identical set of 
questions. This allows for consistency and precision in terms of the wording 
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of the questions, and makes the processing of the answers easier. (Occasion- 
ally, pictures might be used instead of written questions.) 

e Gather information by asking people directly about the points concerned with 
the research. Questionnaires work on the premise that if you want to find 
out something about people and their attitudes you simply go and ask them 
what it is you want to know, and get the information ‘straight from the 
horse’s mouth’. 


There is an assumption in this chapter that the kinds of research questionnaire 
being considered are the postal type and the Internet type — not the face-to- 
face clipboard questionnaire that involves more personal, interactional fac- 
tors. If a clipboard-style questionnaire is being considered, the effect of such 
interactional factors must be considered in addition to many of the points 
related here (see Chapter 10 on Interviews). 


When is it appropriate to use a questionnaire? 


Different methods are better suited to different circumstances, and question- 
naires are no exception. Although they can be used, perhaps ingeniously, 
across a wide spectrum of research situations, questionnaires (both postal and 
Internet) are at their most productive: 


e when used with large numbers of respondents in many locations; 

e when what is required tends to be fairly straightforward information — 
relatively brief and uncontroversial; 

e when there is a need for standardized data from identical questions — without 
requiring personal, face-to-face interaction; 

e when the respondents can be expected to be able to read and understand the 
questions — the implications of age, intellect, language, and eyesight need to 
be considered; 

e when the social climate is open enough to allow full and honest answers. 


What kinds of data are collected by questionnaires? 


Questionnaires rely on written information supplied directly by people in 
response to questions asked by the researcher. In this respect, the kind of data 
is distinct from that which could be obtained from interviews, observation or 
documents. The information from questionnaires tends to fall into two broad 
categories — ‘facts’ and ‘opinions’ — and it is vital that at all stages of using 
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questionnaires the researcher is clear about whether the information being 
sought is to do with facts or to do with opinions. 

Factual information does not require much in the way of judgement or per- 
sonal attitudes on the part of respondents. It just requires respondents to 
reveal straightforward information (such as their address, age, sex, marital 
status or number of children). An example of a ‘fact’ question might be ‘Which 
TV programmes did you watch last night?’ 

Opinions, attitudes, views, beliefs, preferences, etc. can also be investigated 
using questionnaires. In this case, though, respondents are required to reveal 
information about feelings, to express values and to weigh up alternatives in a 
way that calls for a judgement about things rather than the mere reporting of 
facts. An example of an ‘opinion’ question might be ‘Which is your favourite 
TV programme?’ 

It is worth stressing that, in practice, questionnaires are very likely to include 
questions about both facts and opinions. Political opinion polls, for instance, 
might include factual questions about how people actually voted at the 
last election as well as questions about feelings of support for particular polit- 
ical parties’ policies, and market researchers might want to know factual 
information about the age, social class, sex, etc. of the people whose opinions, 
attitudes and preferences they are investigating. 


Planning the use of questionnaires 


Questionnaires tend to be ‘one-offs’. In general, researchers do not have the 
time or resources to repeat pieces of research which involve the use of ques- 
tionnaires; nor do they have the opportunity to make amendments and cor- 
rections to the questionnaire once it has been printed and distributed. And the 
vast majority of respondents are likely to be less than sympathetic to a plea 
from the researcher to fill in the questionnaire a second time in order to over- 
come a mistake in the first version. There is, therefore, a lot of pressure to get it 
right first time. This involves planning. The costs, the timing and other factors 
linked to the distribution of the questionnaire need to be carefully considered 
because each has a direct bearing on the researcher’s prospects of getting it 
right first time. 


Costs 


There are costs associated with the production, distribution and collection of 
the questionnaire and with the analysis of the results. There are printing costs 
to be considered. There might be postal charges and the need for envelopes 
(increased by the need to follow up non-respondents). Will the costs include 
supplying a stamped addressed envelope? Is it the intention to use a computer 
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to analyse the results and, if so, what costs might emerge in terms of the 
necessary software, web hosting services and data preparation? The shrewd 
researcher works out the resource implications before embarking on the use of 
questionnaires. 


Preparation time 


The production of a well-designed questionnaire can take time. First, there 
is the lead-in time of developing the suitable questions and piloting the 
questionnaire. Second, if using a large postal survey, there can be time delays 
arising from the process of printing and production. 


Organization 


The process of distribution, collection and analysis of results from question- 
naires demands an eye on organization. A fundamental skill of good research 
is tight organization, and nowhere is this needed more than in the use of a 
questionnaire survey. From the outset the researcher must keep a record of 
how many questionnaires are sent out, to whom they are sent and when they 
were sent. Unless the questionnaire is anonymous, responses can then be 
checked off against the mailing list and an accurate follow-up survey of non- 
respondents undertaken at an appropriate time. Obviously, a filing system of 
some sort is vital. 


Schedule 


Questionnaires, especially when used as part of a broad postal survey, do 
not supply instant results. The researcher, therefore, needs to consider what 
timescale is likely to be involved before the results from the questionnaire are 
available for analysis. The timescale for the research, it should be stressed, 
involves all the time spent in the production, distribution and collection of 
the questionnaires. It is not just the time between sending out the question- 
naires and receiving the completed returns; the planning should involve the 
whole period between the conception of the research and the receipt of data. 
The researcher needs to be aware of this and plan accordingly. 


Permission 


Depending on the nature of the questionnaire and the people to whom it is 
being sent, there may be the need to gain permission from those in authority 
to conduct the survey. For example, if a questionnaire is being contemplated 
for use with young people and it is intended to distribute the questionnaires 
by hand at schools or youth clubs, staff and the local authorities are likely 
to want to give approval before allowing the survey to proceed using their 
facilities. The researcher needs to be cautious on this score. It is dangerous to 
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short-circuit proper channels of authority. However, gaining permission from 
appropriate authorities can take time and this consideration needs to be built 
into the schedule. 


Internet questionnaires 


Using the Internet, questionnaires can be delivered in one of three main ways. 
With each of these the basic principles of planning and good design remain 
much the same as they do for questionnaires in general. 

An email questionnaire includes the questions as part of the email itself. Such 
questionnaires have the virtue of being simple to construct and easy to answer. 
The respondent only needs to complete the answers and then use the reply 
button to send back the questionnaire. The disadvantages are that: (1) the 
questionnaire may not be completed properly; and (2) the design of the ques- 
tionnaire is necessarily rather ‘basic’. For technical reasons an email question- 
naire cannot be made to look very attractive, since there are limitations on the 
design features that can be incorporated (e.g. no buttons, restricted layout). 

A questionnaire sent as an attachment to an email offers much more by way of 
design features. Using word processing or spreadsheet software it is simple to 
produce and can be made quite attractive. The disadvantage, however, is that 
the reply is not as easily executed as it is with the email questionnaire. The 
respondent needs to open, complete and save an attachment and then 
reattach it to the email reply. 

A web-based questionnaire is designed as a web page and located on a host site 
where visitors to the site can access it. This has two advantages. First, the design 
can take advantage of all the usual web page features that people find attractive 
(buttons, icons, frames). The questionnaire can be colourful and attractive 
and can incorporate graphics. Respondents can select from a predefined range 
of answers and simply ‘submit’ the completed form at one keystroke. The 
researcher can get answers via tick boxes and text-entry boxes. Second, the 
responses can be read automatically into a spreadsheet or database, which has 
the twin benefits of speed and accuracy in terms of data collection. The dis- 
advantages of this approach are that it requires rather more by way of technical 
skill and access to web hosting resources than the other types do, and that it 
relies on people coming to the website. This latter problem can be addressed by 
emailing people telling them about the survey and including in the email a 
hypertext link to the website so that all the respondent needs to do is double 
click on the link in order to go to the website and open the questionnaire. 


Link up with Internet surveys, pp. 13-14 a 
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Vital elements of a research questionnaire 


All research questionnaires should include some background information 
about the research, some instructions to the respondent to help them with the 
task of filling in the answers, and some means of giving each completed ques- 
tionnaire a unique identifying code or number. These are three features whose 
inclusion is considered to be ‘good practice’ in relation to any research 
questionnaire. 


Background information 


From both an ethical and a practical point of view, the researcher needs to 
provide sufficient background information about the research and the ques- 
tionnaire. Each questionnaire should have some information about: 


e The sponsor. Under whose auspices is the research being undertaken? Is it 
individual research or does the research emanate from an institution? 
Headed notepaper is an obvious way to indicate the nature of the institu- 
tion from which the questionnaire comes. 

e The purpose. What is the questionnaire for, and how will the information be 
used? It is important here to reveal sufficient information to explain the 
purpose of the research without going too far and ‘leading’ the respondent 
into a line of answering. A brief paragraph should suffice. 

e Return address and date. It is vital that the questionnaire contains in it some- 
where quite visibly the name and contact address (postal or email) of the 
person(s) to whom the completed questionnaire is to be returned and 
the date by which it is required. 

e Confidentiality. Assuming that the research is operating according to the 
normal code of ethics for social researchers, the information collected 
will not be made publicly available. Respondents should be reassured on 
this point. 

e Voluntary responses. In the vast majority of cases, researchers collect infor- 
mation from respondents who volunteer to co-operate with the research. 
Rewards are not generally used (though in market research there may 
be some enticements offered), and people are not usually obliged to 
respond. Again, this point should be acknowledged on the cover page 
and respondents reassured that the questionnaire is to be completed 
voluntarily. 

e Thanks. It follows from the fact that questionnaires are normally volunta- 
rily completed that the researcher is beholden to those who co-operated. 
Courtesy suggests that some word of thanks from the researcher should 
appear either on the front cover or right at the end of the questionnaire. 
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Instructions to the respondent 


It is very important that respondents are instructed on how to go about 
answering the questions. It is no good assuming that it is self-evident; experi- 
ence will soon prove that wrong. Mistakes that occur can invalidate a whole 
questionnaire, so it is worth being meticulously careful in giving instructions 
on how to complete the answers: 


e Example. It is useful to present an example at the start of the questionnaire. 
This can set the respondent’s mind at rest and indicate exactly what is 
expected of him or her. 

e Instructions. Specific instructions should be given for each question where 
the style of question varies throughout the questionnaire (e.g. put a tick in 
the appropriate box, circle the relevant number, delete as appropriate). 


The allocation of serial numbers 


With web-based questionnaires, the serial numbers are added automatically. 
Other forms of questionnaire might need serial numbers to be added manu- 
ally. In this case, each questionnaire should be numbered so that it can be 
distinguished from others and located if necessary. The serial number can be 
used to identify the date of distribution, the place and possibly the person. But 
note: if the questionnaires are anonymous, such serial numbers should only 
allow general factors such as the date and location to be identified — not the 
identity of any individual person. 


The length of the questionnaire 


There is no hard and fast rule about the number of questions that can 
be included in a questionnaire. This will depend on factors such as the 
topic under investigation, how complex the questions are, the nature of 
the respondents who have been targeted and the time it takes to complete 
the questionnaire. Decisions about the size of a questionnaire are ultimately a 
matter of judgement on the part of the researcher, who needs to gauge how 
many questions can be included before the respondent is likely to run out of 
patience and consign the questionnaire to the waste bin. 

In most cases, researchers do not get a second chance to follow up issues 
they might have missed in the initial questionnaire. Conscious of this ‘one 
shot’ constraint, there may be the temptation to ask about everything that 
might possibly be of relevance. It is, after all, vital to cover all key matters 
if the questionnaire is to supply data which permit a reasonable analysis by 
the researcher. But the shrewd researcher realizes that it is counter-productive 
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to include everything that might feasibly have some relevance to the research 
issues. Every effort should be made to keep the questionnaire as brief as pos- 
sible by restricting the scope of the questions to crucial issues related to the 
research, and avoiding any superfluous detail or non-essential topics. When 
designing a questionnaire, then, the researcher has to walk a tightrope 
between ensuring coverage of all the vital issues and ensuring the question- 
naire is brief enough to encourage people to bother answering it. 
To accomplish this balancing act there are certain rules to bear in mind: 


e Only ask those questions which are absolutely vital for the research. The better 
the research is planned, the easier it will be to identify the absolutely crucial 
questions and discard the ‘just in case I need this information later’ 
questions. 

e Be rigorous in weeding out any duplication of questions. For example, if a 
questionnaire contains as separate questions, ‘What is your date of birth?’ 
and ‘How old are you?’ we need to ask just how vital it is that both 
are included. A moment’s reflection might lead the researcher to the 
conclusion that one or other will supply adequate information for the 
particular purposes of the investigation — or that one can be deduced from 
the other. 

e Make the task of responding to the questionnaire as straightforward and speedy as 
possible. 

e Pilot the questionnaire to see how long it takes to answer and then consider 
whether it is reasonable to expect the specific target group to spare this 
amount of time supplying the answers. 


Good practice: keep questionnaires as short as possible 


A questionnaire needs to be crisp and concise, asking just those questions 
which are crucial to the research. 

It is worth remembering that there is, perhaps, no more effective deterrent 
to answering a questionnaire than its sheer size. 


Devising the questions 


Suitable topics 


The willingness of respondents to co-operate with research by filling in 
a questionnaire can easily be undermined by the nature of the topics in 
the questionnaire. To avoid this possibility, researchers need to ensure that the 
questions are on topics that respondents will: 
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e be willing to answer. When constructing the questions, the researcher needs 
to think about whether the respondent will want to answer the kind of 
questions that are being asked. This point applies, in particular, to questions 
touching on personal matters or sensitive areas linked with political and 
religious beliefs. 

e have some information, knowledge, experience or opinions on. It is no good con- 
structing questions where the responses are likely to be a series of ‘don’t 
knows’ or ‘not applicable’. 


Direct or indirect questions 


Researchers need to consider whether the information being sought from the 
respondent is amenable to direct questions, or whether it will need to be 
measured by indirect means. Where the information being sought is of an 
overt, factual nature, there is not normally a problem on this point. The ques- 
tions will take the form of ‘How many times ...? or ‘When did you last .. .? 
However, questionnaires are frequently geared to less straightforward matters, 
where a direct question would be inappropriate. If, for example, the researcher 
wishes to know what social class the respondent comes from, it would not be 
wise to ask the straightforward question, ‘What is your social class?’ Apart 
from the fact that some people might find this offensive, the term social class 
has different meanings for different people, and the question will produce 
inconsistent answers. In this case, the researcher needs to pinpoint exactly 
what he or she defines as social class and then devise questions that will supply 
the type of information which will allow the respondent’s social class to be 
deduced. For this purpose, indirect questions will be used about occupation, 
income, education, etc. 


The wording of the questions 


Questions should not be irritating or annoying for the respondents to answer. 
The success of the questionnaire, after all, depends on the willingness of the 
respondents to spend their time completing the answers and they should not 
be deterred from this task by any sense of frustration arising from the manner 
in which the questions are worded. There are some specific things that 
researchers can do to help avoid this possibility: 


e Make sure the wording is completely unambiguous. 

e Avoid vague questions. The more specific and concrete the question, the 
easier it is to give a precise answer. And, in all probability, the answer will 
prove to be of more value to the researcher. 

¢ Use only the minimum amount of technical jargon. In terms of a questionnaire, 
the aim is not to see how clever people are. 

e Use wording that is suited to the specific target group. Questions geared to 
14-year-olds will need to be different in terms of conceptual complexity and 
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wording from ones aimed at, for example, members of a professional 
association. 

e Keep the questions as short and straightforward as possible. This will avoid 
unnecessary confusion and wasted time re-reading questions or trying to 
decipher the meaning of the question. 

e Avoid asking the same question twice in a different fashion (except as ‘check’ 
questions). 

e Avoid the use of ‘leading’ questions. These are questions which suggest an 
answer or which prompt the respondent to give a particular kind of answer 
(e.g. Would you agree that there should be controls on the emission of 
carbon dioxide from cars?). 

e Be sure to include sufficient options in the answer. If the list might prove 
too long, selecting the main possibilities and then tagging on the ‘Other 
(please specify)’ option is a good strategy. 

e Pay attention to the way the questions are numbered. Obviously they should be 
sequential, but there are clever ways of using sub-questions (e.g. 4a, 4b and 
so on) which can help the respondent to map his or her way through the 
series of questions. 

e Avoid words or phrases which might cause offence. If, for example, the ques- 
tionnaire involves ethnic minority issues, it is prudent to use the ‘politically 
correct’ terms. 


Good practice: questions need to be worded with clarity and precision 


There is little room for vagueness or imprecision. This means that the 
researcher must have a clear vision of exactly what issues are at the heart 
of the research and what kind of information would enlighten those issues. 


The order of the questions 


The ordering of the questions in a questionnaire is important for two reasons. 
First and foremost, it can entice or deter the respondent from continuing with 
the exercise of providing answers. If the respondent is immediately faced with 
the most complex of the questions at the start of the questionnaire, this might 
deter him or her from going any further. However, if the questionnaire starts 
with straightforward questions and then gradually moves towards such ques- 
tions at a later stage, there is a greater likelihood that the respondent will 
persevere. This same point is true for those questions which might be per- 
ceived as more personal and on sensitive issues. There will be a greater chance 
of success if such questions appear later in the questionnaire than if they 
appear right at the start. 

The second way in which the ordering of questions can be important is that 
questions asked at an earlier point in the questionnaire can affect the answers 
supplied at a later stage. This is evident in many commercial ‘marketing/sales 
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questionnaires’, where there is a conscious attempt to make use of this feature 
of questionnaires. 


Good practice: the order of the questions 

It is usually best to ensure that: 

e the most straightforward questions come at the start; 

e the least contentious questions and least sensitive issues are dealt with 
at the beginning of the questionnaire; 

e the sequence of questions does not lead the respondents towards ‘inevit- 
able’ answers, where the answers to later questions are effectively 
predicated on the answers to earlier ones. 


Types of questions 


There are a variety of ways in which questions can be put in a questionnaire, 
and these are outlined below. However, from the outset it is worth giving some 
thought to whether the overall questionnaire will benefit from using a variety 
of kinds of questions, or whether it is better to aim for a consistent style 
throughout. Variety has two potential advantages. First, it stops the respond- 
ent becoming bored. Second, it stops the respondent falling into a ‘pattern’ of 
answers where, for example, on a scale of 1 to 5 he or she begins to put 4 down 
as the answer to all questions. Aiming for a consistent style of question, for its 
part, has the advantage of allowing the respondent to get used to the kind of 
questions so that they can be answered quickly and with less likelihood of 
confusion or misunderstanding. There is no hard and fast rule on this point: it 
is down to a judgement call on the part of the researcher. 


‘Open’ and ‘closed’ questions 


Open questions are those that leave the respondent to decide the wording of 
the answer, the length of the answer and the kind of matters to be raised in the 
answer. The questions tend to be short and the answers tend to be long. For 
example, the questionnaire might ask, ‘How do you feel about the inclusion 
of nuclear arms as part of Britain’s defence capability?’ and then provide a 
number of empty lines which invite the respondent to enter his or her 
thoughts on the matter. 

The advantage of ‘open’ questions is that the information gathered by way 
of the responses is more likely to reflect the full richness and complexity of the 
views held by the respondent. Respondents are allowed space to express them- 
selves in their own words. Weighed against this, however, there are two dis- 
advantages which are built into the use of open questions. First, they demand 
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more effort on the part of the respondents (which might well reduce their 
willingness to take part in the research). Second, they leave the researcher with 
data which are quite ‘raw’ and require a lot of time-consuming analysis before 
they can be used. 

Closed questions structure the answers by allowing only answers which fit 
into categories that have been established in advance by the researcher. The 
researcher, in this case, instructs the respondent to answer by selecting from a 
range of two or more options supplied on the questionnaire. The options can 
be restricted to as few as two (e.g. ‘Yes’ or ‘No’; ‘Male’ or ‘Female’) or can include 
quite complex lists of alternatives from which the respondent can choose. 

The advantages and disadvantages of the ‘closed’ question are more or less a 
mirror image of those connected with the open, unstructured approach. In a 
nutshell, the main advantage is that the structure imposed on the respond- 
ents’ answers provides the researcher with information which is of uniform 
length and in a form that lends itself nicely to being quantified and compared. 
The answers, in fact, provide pre-coded data that can easily be analysed. 
Weighed against this, there are two disadvantages. First, there is less scope for 
respondents to supply answers which reflect the exact facts or true feelings on 
a topic if the facts or opinions happen to be complicated or do not exactly fit 
into the range of options supplied in the questionnaire. The closed question 
allows for less subtlety in the answers. Second, and largely as a result of the first 
disadvantage, the respondents might get frustrated by not being allowed to 
express their views fully in a way that accounts for any sophistication, intri- 
cacy or even inconsistencies in their views. Third, the questionnaire can be 
evaluated according to its likelihood of achieving a decent response rate. In 
order to get a representative picture, the questionnaire needs to avoid ‘turning 
people off’ by being poorly presented, taking too long to complete, asking 
insensitive questions, etc. 


/\ Caution: closed questions and types of quantitative data 


Closed questions, in general, lend themselves to the production of 
quantitative data. However, it is very important for the researcher to be 
clear at the design stage about which particular kind(s) of quantitative 
data will be collected (nominal, ordinal, interval, etc.) and which spe- 
cific statistical procedures will be used. Unless the researcher thinks 
ahead on this point, there is a very real danger that the data collected 
will turn out to be inappropriate for the kind of analysis that the 
researcher eventually wants to undertake. 


Link up with Quantitative data, Chapter 13 A 
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Example: Nine types of question that can be used in a questionnaire 
1 A statement 
Example: 
What do you think about the UK’s membership of the European Union? 


2 A list 
Example: 
Please list the issues you feel are most important in relation to the UK’s 
membership of the European Union: 


3 A ‘yes/no’ answer 
Example: 
Have you travelled from the UK to another European Union country in the 
past 12 months? Yes/No 


4 Agree/disagree with a statement 
Example: 
Would you agree or disagree with the following statement? 
European unity carries economic advantages which outweigh the political 
disadvantages. Agree/Disagree 


5 Choose from a list of options 
Example: 
Which ONE of the following list of European countries do you feel has the 
strongest economy? 


Spain UK Belgium Netherlands 
Ireland France Germany Italy 


6 Rank order 
Example: 
From the following list of European countries choose the THREE which 
you feel have the strongest economies and put them in rank order: 
1 = strongest, 2 = second strongest, 3 = third strongest. 


Spain UK Belgium Netherlands 
Ireland France Germany Italy 
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7 Degree of agreement and disagreement: the Likert Scale 
Example: 
Membership of the European Union is a good thing for the UK. 


Strongly agree Agree Neutral Disagree Strongly disagree 


8 Rate items 
Example: 
How significant would you rate the following factors in affecting further 
European integration? 


Not Very 
significant significant 
political sovereignty 1 2 8 4 5 6 7 
national identities 1 2 3 4 5 6 7 
past history il 2 3 4 5 6 7 
religiousdifferences il 2 3 4 5 6 7 
language barriers 1 2 3 4 5 6 7 


9 Feelings about a topic: the semantic differential 
Example: 
Progress towards European unity is: 


Important 1 2 3 4 5 Unimportant 
Difficult 1 2 3 4 5 Easy 

Risky 1 2 3 4 5 Safe 
Unlikely 1 2 3 4 5 Likely 
Boring 1 2 3 4 5 Interesting 


Evaluating questionnaires 


There are three basic criteria for evaluating a research questionnaire. The first 
of these concerns an assessment of the likelihood that the questionnaire will 
provide full information on the particular topic(s) of research. The value of the 
questionnaire will depend in no small measure on the extent to which it 
includes coverage of all vital information pertaining to the area of research. 

The second criterion concerns the likelihood that the questionnaire will 
provide accurate information. What confidence do we have that the responses 
on the questionnaires are as full and honest as they can be — free from mis- 
chievous attempts to scupper the research, errors arising through ambiguous 
questions, etc.? 
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Advantages of questionnaires 


© Questionnaires are economical, in the sense that they can supply a consider- 
able amount of research data for a relatively low cost in terms of materials, 
money and time. 

e Relatively easy to arrange. Questionnaires are easier to arrange than, for 
example, personal interviews. There is no need to ‘arrange’ it at all, in fact, 
since the questionnaire may be simply sent unannounced to the respond- 
ent. (Some researchers, though, have sought to improve the response rate 
by contacting respondents before they send a questionnaire to them. This 
contact can be by phone, email or letter.) 

e Questionnaires supply standardized answers, to the extent that all respondents 
are posed with exactly the same questions — with no scope for variation to 
slip in via face-to-face contact with the researcher. The data collected, then, 
are very unlikely to be contaminated through variations in the wording of 
the questions or the manner in which the question is asked. There is little 
scope for the data to be affected by ‘interpersonal factors’. 

e Pre-coded answers. A further, and important, advantage of the questionnaire 
is that it encourages pre-coded answers. As we have seen, this is not an 
essential facet of questionnaires, because unstructured answers can be 
sought. However, particularly with questionnaires, the value of the data is 
likely to be greatest where respondents provide answers that fit into a range 
of options offered by the researcher. These allow the speedy collation and 
analysis of data by the researcher. They also have an advantage for the 
respondents, who, instead of needing to think of how to express their ideas, 
are faced with the relatively easy task of only needing to pick one or more 
answers which are spelt out for them. 

e Data accuracy. Surveys using the Internet can be designed so that the 
data contained in the completed questionnaires can be fed straight into 
a data file, thus automating the process of data entry. This effectively elim- 
inates the human error factor that inevitably arises when people need 
to read the responses to a paper questionnaire and then enter the data 
manually via the computer keyboard. More advanced design techniques 
can ensure that all sections of the Internet questionnaire get completed, 
an advantage even when compared with paper questionnaires that are 
designed to be read automatically via an optical mark reading (OMR) 
machine. 
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Disadvantages of questionnaires 


In many respects the potential disadvantages of questionnaires go hand in 
glove with the potential advantages. You can’t have one without the other. 


e Pre-coded questions can be frustrating for respondents and, thus, deter them from 
answering. The advantage for the researcher of using pre-coded answers set 
out in the questionnaire carries with it a possible disadvantage. While the 
respondents might find it less demanding merely to tick appropriate boxes 
they might, equally, find this restricting and frustrating. So, on the one hand, 
the ‘tick box’ routine might encourage people to respond but, on the other 
hand, this same routine might be experienced as negative and put people 
off co-operating with the research. 


Se, 
Link up with Response rates, pp. 19-22 e 


e Pre-coded questions can bias the findings towards the researcher’s, rather than the 
respondent’s, way of seeing things. Questionnaires, by their very nature, can 
start to impose a structure on the answers and shape the nature of the 
responses in a way that reflects the researcher’s thinking rather than the 
respondent’s. Good research practice will minimize the prospect of this, but 
there is always the danger that the options open to the respondent when 
answering the questions will channel responses away from the respondent’s 
perception of matters to fit in with a line of thinking established by the 
researcher. 

e Questionnaires offer little opportunity for the researcher to check the truthfulness 
of the answers given by the respondents. Because the researcher does not meet 
the respondent and because the answers are given ‘at a distance’, the 
researcher cannot rely on a number of clues that an interviewer might have 
about whether the answers are genuine or not. The interviewer might see 
some incongruity between answers given by the same interviewee and be 
able to probe the matter. Or the interviewer might note a disparity between 
a given answer and some other factor (e.g. stated occupation and apparent 
level of income). In the case of the questionnaire, however, if a respondent 
states their occupation to be a ‘dentist’, it would seem at first glance that the 
researcher has little option but to accept this as true. Likewise, on matters of 
taste or opinion, if the respondent answers along a particular line, the ques- 
tionnaire researcher would seem to have no solid grounds for challenging 
the answer. This is all the more true if the questionnaires are anonymous. 
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Checklist for the use of questionnaires 


When using a questionnaire for research you should feel confident about 
answering ‘yes’ to the following questions: 


Has time been allowed for the planning, design and 
production stages of producing a questionnaire? 


Does the research schedule allow time for respondents 
to complete and return the questionnaire (including follow-ups)? 


Have resources been allocated for all the costs involved 
in the production and distribution of the questionnaire? 


Has the questionnaire been piloted? 

Is the layout clear? 

Is there an explanation of the purpose of the questionnaire? 
Is there a contact address on the questionnaire? 

Have thanks been expressed to the respondents? 


Are there assurances about anonymity and the 
confidentiality of data? 


Have serial numbers been given to the questionnaires? 


Are there clear and explicit instructions on how the 
questions are to be completed? 


Have the questions been checked to avoid any duplication? 


Are the questions clear and unambiguous? 
Have all non-essential questions been excluded? 
Are the questions in the right order? 


Will closed questions produce the required kind of 
numerical data (nominal, ordinal, interval, ratio, discrete)? 


© M. Denscombe, The Good Research Guide. Open University Press. 
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INTERVIEWS 


What is an interview? e When is it appropriate to use interviews? 

e Types of research interview e The interviewer effect e Planning and 
preparation for interviews e Interview skills e Conducting the interview 
e Recording the interview e The validity of interview data: how do you 


know the informant is telling the truth? e Internet interviews 
e Advantages of interviews e Disadvantages of interviews 
e Checklist for the use of interviews 


What is an interview? 


Interviews are an attractive proposition for project researchers. At first glance, 
they do not seem to involve much technical paraphernalia and they draw ona 
skill that researchers already have - the ability to conduct a conversation. The 
reality, though, is not quite so simple. Although there are a lot of superficial 
similarities between a conversation and an interview, interviews are actually 
something more than just a conversation. Interviews involve a set of assump- 
tions and understandings about the situation which are not normally associ- 
ated with a casual conversation (Denscombe 1983; Silverman 1985). When 
someone agrees to take part in a research interview: 


e There is consent to take part. From the researcher’s point of view this is par- 
ticularly important in relation to research ethics. The interview is not done 
by secret recording of discussions or the use of casual conversations as 
research data. It is openly a meeting intended to produce material that will 
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be used for research purposes — and the interviewee understands this and 
agrees to it. 

e Interviewees’ words can be treated as ‘on the record’ and ‘for the record’. It is, of 
course, possible for interviewees to stipulate that their words are not to be 
attributed to them, or not to be made publicly available. The point is, 
though, that unless interviewees specify to the contrary, the interview talk 
is ‘on record’ and ‘for the record’. 

e The agenda for the discussion is set by the researcher. Although the degree 
of control exercised by the researcher will vary according to the style of 
interviewing, there is a tacit agreement built into the notion of being inter- 
viewed that the proceedings and the agenda for the discussion will be 
controlled by the researcher. 


/\ Caution: interviews are not the same as a conversation 


The superficial similarity between an interview and a conversation can 
generate an illusion of simplicity. We all have conversations and it is 
likely that most of us do not have too much difficulty with them. So an 
interview should be fairly straightforward. As long as we know to whom 
we are going to talk and what we want to ask, the rest should be plain 
sailing. Here lies the problem. The researcher can be lulled into a false 
sense of security. The superficial similarity can encourage a relaxed 
attitude to the planning, preparation and conduct of the method that 
would be unlikely were it to involve questionnaires or experiments. In 
reality, interviewing is no easy option. It is fraught with hidden dangers 
and can fail miserably unless there is good planning, proper prepar- 
ation and a sensitivity to the complex nature of interaction during the 
interview itself. 


When is it appropriate to use interviews? 


Although interviews can be used for the collection of straightforward factual 
information, their potential as a data collection method is better exploited 
when they are applied to the exploration of more complex and subtle phe- 
nomena. If the researcher wants to collect information on simple and uncon- 
troversial facts, then questionnaires might prove to be a more cost-effective 
method. But when the researcher needs to gain insights into things such as 
people’s opinions, feelings, emotions and experiences, then interviews will 
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almost certainly provide a more suitable method — a method that is attuned 
to the intricacy of the subject matter. To be specific, interviews - in particular 
in-depth interviews — lend themselves to the collection of data based on: 


e Opinions, feelings, emotions and experiences. The nature of these means that 
they need to be explored in depth and in detail rather than simply reported 
in a word or two. 

e Sensitive issues. When the research covers issues that might be considered 
sensitive or rather personal there is a case to be made for using interviews. 
Using a careful and considerate approach, participants can be encouraged to 
discuss personal and sensitive issues in open and honest manner. 

e Privileged information. Here, the justification for interviews is based on the 
value of contact with key players in the field who can give privileged infor- 
mation. The depth of information provided by interviews can produce best 
‘value for money’ if the informants are willing and able to give information 
that others could not - when what they offer is an insight they have as 
people in a special position ‘to know’. 


The decision to use interviews for a research project, of course, needs to take 
account of their feasibility as a data collection method. Before embarking on a 
programme of interviews the researcher needs to feel assured that: 


e It is possible to gain direct access to the prospective interviewees. There is obvi- 
ously no point in pursuing the idea of conducting interviews unless there 
are good grounds for believing that the necessary people can be accessed, 
and that some agreement can be obtained from all the parties involved in 
the research. 

e The interviews are viable in terms of the costs in time and travel involved. With 
limited resources, the researcher needs to ensure that the people are not 
distributed too widely across a large geographical area and that conducting 
the interviews will not incur prohibitive costs. 


Types of research interview 


Structured interviews 


Structured interviews involve tight control over the format of the questions 
and answers. In essence, the structured interview is like a questionnaire which 
is administered face-to-face with a respondent. The researcher has a predeter- 
mined list of questions, to which the respondent is invited to offer limited- 
option responses. The tight control over the wording of the questions, the 
order in which the questions occur and the range of answers that are on offer 
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have the advantage of ‘standardization’. Each respondent is faced with identi- 
cal questions. And the range of pre-coded answers on offer to respondents 
ensures that data analysis is relatively easy. The structured interview, in this 
respect, lends itself to the collection of quantitative data. 

Structured interviews are often associated with social surveys where 
researchers are trying to collect large volumes of data from a wide range of 
respondents. Here, we are witnessing the replacement of interviewers armed 
with clipboards and paper questionnaires with those using laptop computers 
to input information direct into a suitable software program. Such computer- 
assisted personal interviewing (CAPI) has the advantage of using software with 
built-in checks to eliminate errors in the collection of data, and it allows 
quick analysis of the data. However, its relatively large initial costs, caused by 
the purchase of the laptop computers, the development of suitable software 
and the training involved, mean that CAPI is better suited to large-budget, 
large-number surveys than to small-scale research. 


ee 
Link up with Questionnaires, Chapter 9 ° 


Whether using a laptop computer, telephone or printed schedule of ques- 
tions, the structured interview, in spirit, bears more resemblance to question- 
naire methods than to the other types of interviews. For this reason, this 
chapter focuses on semi-structured and unstructured interviews rather than structured 
interviews. 


Semi-structured interviews 


With semi-structured interviews, the interviewer still has a clear list of issues to 
be addressed and questions to be answered. However, with the semi-structured 
interview the interviewer is prepared to be flexible in terms of the order in 
which the topics are considered, and, perhaps more significantly, to let the 
interviewee develop ideas and speak more widely on the issues raised by 
the researcher. The answers are open-ended, and there is more emphasis on the 
interviewee elaborating points of interest. 


Unstructured interviews 


Unstructured interviews go further in the extent to which emphasis is placed 
on the interviewee’s thoughts. The researcher’s role is to be as unintrusive 
as possible — to start the ball rolling by introducing a theme or topic and 
then letting the interviewee develop their ideas and pursue their train of 
thought. 

Semi-structured and unstructured interviews are really on a continuum and, 
in practice, it is likely that any interview will slide back and forth along the 
scale. What they have in common, and what separates them from structured 
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interviews, is their willingness to allow interviewees to use their own words 
and develop their own thoughts. Allowing interviewees to ‘speak their minds’ 
is a better way of discovering things about complex issues and, generally, semi- 
structured and unstructured interviews have as their aim ‘discovery’ rather 
then ‘checking’. 


One-to-one interviews 


The most common form of interview is the one-to-one variety which involves 
a meeting between one researcher and one informant. One reason for its 
popularity is that it is relatively easy to arrange. Only two people’s diaries 
need to coincide. Another advantage is that the opinions and views expressed 
throughout the interview stem from one source: the interviewee. This makes 
it fairly straightforward for the researcher to locate specific ideas with spe- 
cific people. A third advantage is that the one-to-one interview is relatively 
easy to control. The researcher only has one person’s ideas to grasp and 
interrogate, and one person to guide through the interview agenda. And 
a fourth advantage of conducting one-to-one interviews becomes evident 
when the researcher embarks on transcribing the interview tape: it is far 
easier to transcribe a recorded interview when the talk involves just one 
interviewee. There is only one voice to recognize and only one person talking 
at a time. 


Group interviews 


A disadvantage of the one-to-one interview is that it limits the number of 
views and opinions available to the researcher. Listening to one person at a 
time effectively restricts the number of voices that can be heard and the range 
of views that can be included within a research project. Group interviews, 
however, provide a practical solution to this. By interviewing more than one 
person at a time the researcher is able to dramatically increase the number and 
range of participants involved in the research. 

A group interview can be undertaken very much like a one-to-one interview 
in the sense that the interviewer remains the focal point of the interaction that 
takes place. The questions and answers are channelled through the inter- 
viewer. The difference is that instead of each question prompting a response 
from just one interviewee the researcher can get perhaps four responses from 
four people during the interview. 

Increasing the numbers involved can have benefits in terms of the represen- 
tativeness of the data. The inclusion of more participants is likely to mean that 
a broader spectrum of people are covered by the research and that there might 
be a greater variety of experiences and opinions emerging from the investiga- 
tion. Indeed, under certain circumstances researchers can deliberately select 
participants who are very different in order to gather widely differing views 
and experiences on the topic of the interview. 
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But, does this really constitute a group interview? Where the interviewer 
remains the focal point of the communication it suggests that the interview is 
actually conducted with a series of individuals who happen to have been 
collected together at one time and in one place to suit the convenience of 
the researcher and his/her wish to boost the number of interviewees in the 
research. The participants are a group in so far as they are grouped together, 
but the researcher does not interact with participants as a group - only as 
individuals. 

An alternative vision of the group interview is one that stresses the ‘group’ 
characteristics of the interaction during an interview. It sees the group inter- 
view as distinctive in the way that it can get the participants to respond as part 
of a group, rather than as individuals. The researcher’s incentive for using 
a group interview, in this case, is not a quantitative one concerned with 
increased numbers and improved representativeness. It is, instead, a qualita- 
tive one concerned with the way that group discussions can be more illuminat- 
ing. The group discussion allows participants to listen to alternative points of 
view. It allows members to express support for certain views and to challenge 
views with which they disagree. The group interview, in this sense, trades 
on group dynamics. It uses the social and psychological aspects of group 
behaviour to foster the ability of participants to get involved, speak their 
minds and reflect on the views of others. 


Focus groups 


Focus groups consist of small groups of people who are brought together by a 
‘moderator’ (the researcher) to explore attitudes and perceptions, feelings and 
ideas about a specific topic. Typically they last for 1% to 2 hours and are useful 
for gauging the extent to which there are shared views among a group of 
people in relation to a specific topic. Ideally, focus groups have six to nine 
people in them. This is a large enough number to allow a range of views and 
opinions to be present among the group but not too large as to be unmanage- 
able in terms of the discussion. In small-scale research projects the numbers are 
often smaller. The reason for this is that focus groups can be costly and time- 
consuming to arrange. It is not easy to organize a venue for the meeting and 
get six or more people to turn up on time. Nor is it necessarily inexpensive if 
the researcher needs to fund travel and pay for the room. 

Focus groups make particular use of group dynamics and have three distinct- 
ive features: 


e there is a focus to the session, with the group discussion being based on an 
item or experience about which all participants have similar knowledge; 

e particular emphasis is placed on the interaction within the group as a means of 
eliciting information; 

e the moderator’s role is to facilitate the group interaction rather than lead the 
discussion. 
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Link up with Focus groups, Appendix 4 
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Internet interviews 


Interviews and focus groups can be conducted using the Internet. There is an 
obvious advantage in terms of eliminating the time and costs of travelling to 
meet interviewees face to face, and the technology is now easily available. 
Using Skype, for instance, it is possible to conduct face-to-face interviews with 
anyone who has an Internet connection at virtually no cost — provided, of 
course, that the interviewee has the required software. Using a webcam means 
that these interviews can include visual contact with the interviewee. There 
are some practical issues concerned with online interviews and there are, 
indeed, alternative ways of conducting them. These are outlined in a separate 
section later in this chapter. Conducting focus groups online can prove to be 
rather more difficult in terms of the necessary software and these issues are 
considered in Appendix 4. 


The interviewer effect 


Personal identity 


Research on interviewing has demonstrated fairly conclusively that people 
respond differently depending on how they perceive the person asking the 
questions. In particular, the sex, the age and the ethnic origins of the interviewer 
have a bearing on the amount of information people are willing to divulge and 
their honesty about what they reveal. The data, in other words, are affected by 
the personal identity of the researcher. This point applies whether the inter- 
view is conducted face to face or online. In the case of online interviews, 
although interviewees might not be able to see the interviewer, their perception 
of the person who is collecting the data can still influence their willingness to 
divulge information. 

The impact of the researcher’s personal identity, of course, will depend on 
who is being interviewed. It is not, strictly speaking, the identity in its own 
right that affects the data, but what the researcher’s identity means as far as the 
person being interviewed is concerned. Interviewees, and interviewers come to 
that, have their own preferences and prejudices, and these are likely to have some 
impact on the chances of developing rapport and trust during an interview. 

The effect of the researcher’s identity, in practice, will also depend on the 
nature of the topic being discussed. On sensitive issues or on matters regarded 
as rather personal, the interviewer’s identity assumes particular importance. 
If the research is dealing with religious beliefs, with earnings, with sexual 
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relationships, with personal health or any of a host of similar issues, the sex, 
age and ethnicity of the interviewer in relation to the sex, age and ethnicity 
of the interviewee are very likely to influence the nature of the data that 
emerge — their fullness and their honesty. On some questions people can 
be embarrassed. They can feel awkward or defensive. Whenever this is the 
case, there is the possibility that interviewees might supply answers which 
they feel fit in with what the researcher expects from them - fulfilling the 
perceived expectations of the researcher. Or the answers might tend to be 
tailored to match what the interviewee suspects is the researcher’s point 
of view, keeping the researcher happy. Either way, the quality of the data 
suffers. 

From the perspective of the small-scale project researcher there is a limit 
to what can be done about this. Although interviews conducted over the 
Internet, and to a lesser extent interviews conducted by phone, do provide 
some possibilities for disguising who we are, there are definite limits to the 
extent that researchers can disguise their ‘self’ during face-to-face interviews. 
We bring to such interviews certain personal attributes which are ‘givens’ 
and which cannot be altered on a whim to suit the needs of the research 
interview. Our sex, our age, our ethnic origin, our accent, even our occupational 
status, all are aspects of our ‘self’ which, for practical purposes, cannot be changed. 
We can make efforts to be polite and punctual, receptive and neutral, in 
order to encourage the right climate for an interviewee to feel comfortable 
and provide honest answers. What we cannot do is change these personal 
attributes. 


Good practice: being aware of the interviewer effect 
Social researchers should consider the following questions: 


e What are the social status, educational qualifications and professional 
expertise of the people to be interviewed, and how do these compare with 
my own? Is this likely to affect the interviewer—interviewee relationship in a 
positive or negative manner? 

e ls there likely to be an age gap between myself and the interviewee(s) and, 
if so, how might this affect the interview? 

e In relation to the topic being researched, will interviewing someone of the 
opposite sex or a different ethnic group have an impact on their willingness 
to answer questions openly and honestly? 


Self-presentation 


Conventional advice to researchers has been geared to minimizing the impact 
of researchers on the outcome of the research by having them adopt a passive 
and neutral stance. The idea is that the researcher: 
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e presents himself or herself in a light which is designed not to antagonize or 
upset the interviewee (conventional clothes, courtesy, etc.); 

e remains neutral and non-committal on the statements made during the 
interview by the interviewee. 


Passivity and neutrality are the order of the day. The researcher’s ‘self’, adopt- 
ing this approach, is kept firmly hidden beneath a cloak of cordiality and 
receptiveness to the words of the interviewee. To a certain degree, this is sound 
advice. The researcher, after all, is there to listen and learn, not to preach. The 
point is to get the interviewee to open up, not to provoke hostility or put the 
interviewee on the defensive. 


Personal involvement 


One line of reasoning argues that a cold and calculating style of interviewing 
reinforces a gulf between the researcher and the informant, and does little 
to help or empower the informant. Now, if the aims of the research are specif- 
ically to help or empower the people being researched, rather than dispassion- 
ately learn from them, then the approach of the interviewer will need to 
alter accordingly (Oakley 1981). Under these circumstances, the researcher 
will be inclined to show emotion, to respond with feeling and to engage 
in a true dialogue with the interviewee. The researcher will become fully 
involved as a person with feelings, with experiences and with knowledge that 
can be shared with the interviewee. A word of warning, though. This style 
of interviewing remains ‘unconventional’, and the researcher needs to be 
confident and committed to make it work. The researcher also needs to feel 
sure that his or her audience understand and share the underlying logic of 
the approach rather than expecting the researcher to adopt the cool and 
dispassionate stance. 


Planning and preparation for interviews 


The topics for discussion 


With the use of unstructured interviews, it might be argued that the researcher 
should not have preconceived ideas about the crucial issues and direction the 
interview should take. In practice, however, it is not very often that researchers 
operate at the extreme end of the continuum with unstructured interviews. 
In the vast majority of cases, researchers approach an interview with some 
agenda and with some game-plan in mind. In such cases it would be tempting 
fate to proceed to a research interview without having devoted considerable 
time to thinking through the key points that warrant attention. This does 
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not necessarily mean that the researcher needs to have a rigid framework 
of questions and issues in mind — though this will be the case when using 
structured interviews. It does mean that there is likely to be more benefit 
from the interview if he or she is well informed about the topic and has done 
the necessary homework on the issues that are likely to arise during the 
interview. 


Choice of informants 


In principle, there is nothing to stop researchers from selecting informants 
on the basis of random sampling. In practice, though, this is unlikely to 
happen. Interviews are generally conducted with lower numbers than would 
be the case with questionnaire surveys, and this means that the selection 
of people to interview is more likely to be based on non-probability sampling. 
People tend to be chosen deliberately because they have some special contri- 
bution to make, because they have some unique insight or because of the 
position they hold. It is worth emphasizing, though, that there is no hard 
and fast rule on this. It depends on whether the overall aim of the research 
is to produce results which are generalizable (in which case the emphasis 
will be on choosing a representative sample of people to interview) or the 
aim is to delve in depth into a particular situation with a view to exploring 
the specifics (in which case the emphasis will be on choosing key players in 
the field). 

In the case of group interviews, researchers can decide to select inter- 
viewees in order to get a cross-section of opinion within the group, or, 
perhaps, to ensure that group members hold opposing views on the topic for 
discussion. 


Authorization 


In many, if not most, research situations, it will be necessary to get approval 
from relevant ‘authorities’. This will be necessary in any instance of research 
where the people selected to participate in the interviews are either: 


e working within an organization where they are accountable to others higher 
up the chain of command; 

e or potentially vulnerable, and therefore protected by responsible others — the 
young, the infirm and some other groups are under the protection of others 
whose permission must be sought (e.g. school children). 


Organizations, authorities and Internet moderators who grant permission 
will wish to be persuaded that it is bona fide research, and they will also be 
influenced by the personal/research credentials of the researcher. Letters of 
contact, therefore, should spell out the range of factors which will persuade 
the organization or authority that the researcher is both (1) trustworthy and 
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(2) capable. Research which can call on suitable referees or which will be 
conducted under the auspices of a suitable organization (e.g. a university) is at 
an advantage for these. 

To emphasize what ought to be obvious, such authorization to conduct the 
interviews must be gained before the interviews take place. 


Arranging the venue 


In the case of face-to-face interviews, securing an agreement to be interviewed 
is often easier if the prospective interviewee is contacted in advance. This also 
allows both parties to arrange a mutually convenient time for the interview. At 
this point, of course, the researcher will probably be asked how long the inter- 
view will take, and should therefore be in a position to respond. It is most 
unlikely that busy people will feel comfortable with a suggestion that the 
interview will ‘take as long as it takes’. The researcher needs to make a bid for 
an agreed length of time whether it be 15 minutes, half an hour, 45 minutes or 
an hour. 

Where a face-to-face interview takes place ‘on site’, the researcher cannot 
always control events as they might want. This means there is an added danger 
that things can go wrong. Through whatever means, though, the researcher 
needs to try to get a location for the interview which will not be disturbed, which 
offers privacy, which has fairly good acoustics and which is reasonably quiet. 
This can prove to be a pretty tall order in places like busy organizations, 
schools, hospitals, and so on. But at least the desirability of such a venue 
should be conveyed to the person arranging the interview room. 

Within the interview room, it is important to be able to set up the seat- 
ing arrangements in a way that allows comfortable interaction between the 
researcher and the interviewee(s). In a one-to-one interview the researcher 
should try to arrange seating so that the two parties are at a 90 degree angle to 
each other. This allows for eye contact without the confrontational feeling 
arising from sitting directly opposite the other person. With group interviews, 
it is important to arrange the seating to allow contact between all parties with- 
out putting the researcher in a focal position and without hiding individuals 
at the back of the group or outside the group. 


Interview skills 


e The good interviewer needs to be attentive. This may sound obvious, but it is all 
too easy to lose the thread of the discussion because the researcher needs to 
be monitoring a few other things while listening closely to what the 
informant has to say: writing the field notes, looking for relevant non- 
verbal communication, checking that the recorder is working. 
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The good interviewer is sensitive to the feelings of the informant. This is not just a 
matter of social courtesy, though that is certainly a worthy aspect of it. It is 
also a skill which is necessary for getting the best out of an interview. Where 
the interviewer is able to empathize with the informant and to gauge the 
feelings of the informant, they will be in a better position to coax out the 
most relevant information. 

The good interviewer is able to tolerate silences during the talk, and knows 
when to shut up and say nothing. Anxiety is the main danger. Fearing 
that the interview might be on the verge of breaking down, the researcher 
can feel the need to say something quickly to kick-start the discus- 
sion. Worrying about cramming in every possible gem of wisdom in the 
allotted time, the interviewer can be inclined to rush the informant on 
quickly to the next point. But, most of all, feeling uncomfortable when 
the conversation lapses into silence, the interviewer can be all too quick 
to say something when a more experienced interviewer would know that 
the silence can be used as a wonderful resource during interviews (see 
below). 

The good interviewer is adept at using prompts. Although silences can be pro- 
ductive, the interviewer needs to exercise judgement on this. There are 
times during an interview when the researcher might feel that it is necessary 
to spur the informant to speak. Listed below are some examples of how this 
can be done. What the examples share is a degree of subtlety. It is not 
normally acceptable for research interviewers to demand that the informant 
answers the questions. Research interviews are not police interviews. The 
idea is to nudge the informant gently into revealing their knowledge or 
thoughts on a specific point. 

The good interviewer is adept at using probes. There are occasions during 
an interview when the researcher might want to delve deeper into a topic 
rather than let the discussion flow on to the next point. An informant 
might make a point in passing which the researcher thinks should be 
explored in more detail. Some explanation might be called for, or some 
justification for a comment. Some apparent inconsistency in the inform- 
ant’s line of reasoning might be detected, an inconsistency which needs 
unravelling. Examples of how this can be done are listed below. Again, they 
attempt to be subtle and avoid an aggressive stance. 

The good interviewer is adept at using checks. One of the major advantages of 
interviews is that they offer the researcher the opportunity to check that he 
or she has understood the informant correctly. As an ongoing part of the 
normal talk during interviews, the researcher can present a summary of 
what they think the informant has said, which the informant can then 
confirm as an accurate understanding, or can correct if it is felt to be a 
misunderstanding of what has been said. Such checks can be used at stra- 
tegic points during the interview as a way of concluding discussion on one 
aspect of the topic. 

With focus groups, the good facilitator manages to let everyone have a say. 
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It is vital to avoid the situation where a dominant personality hogs the 
discussion and bullies others in the group to agree with his or her opinion. 

e The good interviewer is non-judgemental. As the researcher enters the interview 
situation they should, as far as is possible, suspend personal values and 
adopt a non-judgemental stance in relation to the topics covered during the 
interview. This means not only biting your lip on occasion, but also taking 
care not to reveal disgust, surprise or pleasure through facial gestures. The 
good researcher must also respect the rights of the interviewee. This means 
accepting if a person simply does not wish to tell you something and know- 
ing when to back off if the discussion is beginning to cause the interviewee 
particular embarrassment or stress. This is a point of personal sensitivity 
and research ethics. 


Good practice: tactics for interviews — prompts, probes and checks 


Remain silent 
Repeat the question prompt 
Repeat the last few words spoken by the interviewee prompt 


(prompt 
( 
( 
Offer some examples (prompt 
( 
( 
( 


) 
) 
) 
) 


Ask for an example probe) 
Ask for clarification probe) 
Ask for more details probe) 
Summarize their thoughts (check) 
(‘So, if | understand you correctly .. . 

What this means, then, is that. . .’) 


Conducting the interview 


In the intensity of a research interview it is not easy to attend to all the points 
that should be remembered and, in any case, interviews are ‘live’ events which 
require the interviewer to adjust plans as things progress. Nevertheless, there 
are some pretty basic formalities that need to be observed. There are also a 
number of skills the researcher should exercise and, despite the fluid nature 
of interviews, it is worth spelling out a list of things that go towards a good 
interview. 


Introduction and formalities 


At the beginning there should be the opportunity to say ‘Hello’, to do some 
introductions, to talk about the aims of the research and to say something 
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about the origins of the researcher’s own interest in the topic. During the 
initial phase, there should also be confirmation that you have permission to 
record the discussion and reassurances about the confidentiality of comments 
made during the interview. The aim is to set the tone for the rest of the 
interview - normally a relaxed atmosphere in which the interviewee feels 
free to open up on the topic under consideration. Trust and rapport are the 
keywords. 

During the pre-interview phase, the interviewer should do two other things: 


e prepare the recording equipment; 
e as far as possible, arrange the seating positions to best advantage. 


Starting the interview 


The first question takes on a particular significance for the interview. It should 
offer the interviewee the chance to settle down and relax. For this reason 
it is normally good practice to kick off with an ‘easy’ question: something on 
which the interviewee might be expected to have well-formulated views and 
something that is quite near the forefront of their mind. Two tactics might 
help here. 


e Ask respondents, in a general way, about themselves and their role as it 
relates to the overall area of the interview. This allows the researcher to 
collect valuable background information about informants while, at the same 
time, letting informants start off by covering familiar territory. 

e Use some ‘trigger’ or ‘stimulus’ material, so that the discussion can relate to 
something concrete, rather than launch straight into abstract ideas. 


Monitoring progress 


During the interview, the researcher should keep a discreet eye on the time. 
The good researcher needs to wind things up within the allotted time and will 
have covered most of the key issues during that time. While doing this, the 
good interviewer also needs to attend to the following things during the 
progress of the interview itself: 


e Identify the main points being stated by the interviewee and the priorities 
as expressed by the interviewee. With focus groups, what consensus is 
emerging about the key points? 

e Look for the underlying logic of what is being said by the informant. The 
interviewer needs to ‘read between the lines’ to decipher the rationale lying 
beneath the surface of what is being said. The interviewer should ask ‘What 
are they really telling me here?’ and, perhaps more significantly, ‘What are 
they not mentioning?’ 

e Look for inconsistencies in the position being outlined by the interviewee. 
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If such inconsistencies exist, this does not invalidate their position. Most 
people have inconsistencies in their opinions and feelings on many topics. 
However, such inconsistencies will be worth probing as the interview pro- 
gresses to see what they reveal. 

e Pick up clues about whether the informant’s answers involve an element of 
boasting or are answers intended to please the interviewer. 

e Be constantly on the look-out for the kind of answer that is a ‘fob-off’. 

e Geta feel for the context in which the discussion is taking place. The priorities 
expressed by the interviewee might reflect events immediately prior to 
the interview, or things about to happen in the near future. They might be 
‘issues of the moment’, which would not assume such importance were the 
interview to be conducted a few weeks later. The researcher needs to be 
sensitive to this possibility and find out from the interviewee if there are 
events which are influencing priorities in this way. 

e Keep a suitable level of eye contact throughout the interview and make a 
note of non-verbal communication which might help a later interpretation of 
the interview talk. 


Finishing the interview 


Interviews can come to an end because the interviewee has run out of 
things to say and the interviewer cannot elicit any more information from 
the person. This is not a good state of affairs unless the interview has 
no outside time limit. It is better for the interview to come to a close in 
some orderly fashion guided by the interviewer. Having kept an eye on the 
time, and having ensured that most of the required areas for discussion 
have been covered, the interviewer should draw events to a close making 
sure that: 


e the interviewee is invited to raise any points that they think still need to be 
covered and have not been covered so far; 

e the interviewee is thanked for having given up the time to participate in the 
interview. 


Recording the interview 


The researcher wishing to capture the discussion that happens during the 
interview can rely on memory. However, the human memory is rather unreli- 
able as a research instrument. As psychologists tell us, human memory is 
prone to partial recall, bias and error. Interviewers, instead, can call on other 
more permanent records of what was said. 
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Field notes 


Under certain circumstances researchers will need to rely on field notes written 
soon after the interview or actually during the interview. Sometimes inter- 
viewees will decline to be recorded. This means that what was actually said will 
always remain a matter of recollection and interpretation. There will never 
be an objective record of the discussion. This suits the needs of certain inter- 
viewees, particularly where the discussion touches on sensitive issues, com- 
mercially, politically or even personally. Notes taken during the interview, 
however, offer a compromise in such situations. The interviewer is left with 
some permanent record of their interpretation of what was said, and can refer 
back to this at various later stages to refresh the memory. The notes also act as 
some form of permanent record. However, from the interviewee’s point of 
view, it is always possible to deny that certain things were said and to argue 
that the researcher might have ‘misinterpreted’ a point should the interviewee 
wish to dissociate himself/herself from the point at some later date. A crucial 
advantage of taking field notes at an interview, however, is that they can fill in 
some of the relevant information that a recording alone might miss. Field 
notes can cover information relating to the context of the location, the cli- 
mate and atmosphere under which the interview was conducted, clues about 
the intent behind the statements and comments on aspects of non-verbal 
communication as they were deemed relevant to the interview. 


Good practice: making field notes 


Field notes need to be made during the interview itself or, if this is not feas- 
ible, as soon afterwards as possible. They need to be made while events are 
fresh in the mind of the interviewer. 


Audio recording 


In practice, most research interviewers rely on audio recordings backed up by 
written field notes. Initially, interviewees can feel rather inhibited by the process 
of recording but most participants become more relaxed after a short while. 
When used sensitively, audio recording does not pose too much of a disturb- 
ance to interview situations, and it has certain clear benefits. Audio recordings 
offer a permanent record and one that is fairly complete in terms of the speech 
that occurs. They also lend themselves to being checked by other researchers. 
However, the downside is that they capture only speech, and miss non-verbal 
communication and other contextual factors. Video recordings, for their part, 
capture non-verbal as well as verbal communications and offer a more com- 
plete record of events during the interview. And, as with audio recordings, 
they provide a permanent record that can be checked by other researchers. The 
use of video recordings, however, tends to be the exception rather than the 
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rule. It is not the cost factor that explains this because video equipment is not 
particularly expensive. Generally, it is the intrusiveness of video recordings 
that deters researchers from using them. For practical purposes, most inter- 
viewers would consider that audio recordings provide sufficient data, data that 
are good enough for the purpose of research, and that benefits gained through 
the video recording of an interview are outweighed by the extra disruption 
that video recording brings to the setting. 


Link up with Transcribing audio recordings of interviews, p. 277 j A 
@e@ 


Good practice: recording equipment 


e Use equipment which is good enough to supply adequate sound (or visual) 
reproduction. 

e Be certain that the equipment is functioning well before the interview. 

e Have a reliable power source plus back-up in case of emergency. 

e Choose storage devices that have enough memory, or last long enough, to 
cover the planned duration of the interview without the need to ‘reload’ the 
recorder. 


The validity of interview data: how do you know the 
informant is telling the truth? 


This is a crucial question facing the researcher who uses interview data. When 
the interview is concerned with gathering information of a factual nature, the 
researcher can make some checks to see if the information is broadly corrobor- 
ated by other people and other sources. When the interview concerns matters 
such as the emotions, feelings and experiences of the interviewee, it is a 
lot more difficult to make such checks. Ultimately, there is no absolute 
way of verifying what someone tells you about their thoughts and feelings. 
Researchers are not ‘mind readers’. But there are still some practical checks 
researchers can make to gauge the credibility of what they have been told. It 
should be stressed, though, that these are not watertight methods of detecting 
false statements given during interviews. They are practical ways of helping 
the researcher to avoid being a gullible dupe who accepts all that he or she is 
told at face value. They help the researcher to ‘smell a rat’. By the same token, if 
the following checks are used, the researcher can have greater confidence in 
the interview data, knowing that some effort has been made to ensure the 
validity of the data. 
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Check the data with other sources 


The researcher should make efforts to corroborate the interview data with 
other sources of information on the topic. Triangulation should be used. 
Documents and observations can provide some back-up for the content of the 
interview, or can cast some doubt on how seriously the interview data should 
be taken. Interview content can even be checked against other interviews 
to see if there is some level of consistency. The point is that interview data 
should not be taken at face value if it is at all possible to confirm or dispute the 
statements using alternative sources. 


Link up with Triangulation, Appendix 3 A 


Checking the transcript with the informant 


Where possible, the researcher should go back to the interviewee with the 
transcript to check with that person that it is an accurate statement. Now, of 
course, checking for accuracy is not strictly what is going on here. Unless the 
interviewer also sends a copy of the recording, the interviewee has no way of 
knowing if what appears on the transcript is actually what was said. The point 
of the exercise is more to do with ‘putting the record straight’. If the researcher 
is solely concerned with gathering facts from the interview, this is an oppor- 
tunity to ensure that the facts are correct and that the interviewer has got the 
correct information. That is a nice safeguard. If, alternatively, the interview is 
concerned with a person’s emotions, opinions and experiences, the exercise 
invites the interviewee to confirm that what was said at the time of the inter- 
view was what was really meant, and not said ‘in the heat of the moment’. 
Either way, there is an initial check on the accuracy of the data. 


Check the plausibility of the data 


Some people are interviewed specifically because they are in a position to 
know about the things that interest the researcher. The ‘key players’ are 
picked out precisely because they are specialists, experts, highly experienced — 
and their testimony carries with it a high degree of credibility. This is not 
necessarily the case with those chosen for interview on some other grounds. 
When assessing the credibility of information contained in an interview, 
the researcher needs to gauge how far an informant might be expected to 
be in possession of the facts and to know about the topic being discussed. 
The researcher should ask if it is reasonable to suppose that such a person 
would be in a position to comment authoritatively on the topic — or is there a 
chance that they are talking about something of which they have little 
knowledge? 
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Look for themes in the transcript(s) 


Where possible, avoid basing findings on one interview — look for themes 
emerging from a number of interviews. Where themes emerge across a number 
of interviews, the researcher does not have to rely on any one transcript as the 
sole source of what is ‘real’ or ‘correct’. A recurrent theme in interviews indi- 
cates that the idea/issue is something which is shared among a wider group, 
and therefore the researcher can refer to it with rather more confidence than 
any idea/issue which stems from the words of one individual. 


Link up with Qualitative data, Chapter 14 E 


Internet interviews 


Internet interviews can be conducted with anyone who has access to a com- 
puter with a webcam, who is online and who has communication software like 
Skype installed. The costs are negligible and this mode of conducting inter- 
views allows the researcher to interview people across the world without 
worrying about the time and costs of travel. This is obviously an attractive 
proposition. The interviews take place in real time and, include visual contact 
between the interviewer and the interviewee. The same can be said of video- 
conferencing links but here the costs of the equipment and communication 
can prove to be substantial. 

There are other ways of conducting interviews online which are low cost. 
At its simplest an ‘online interview’ can consist of an exchange of email cor- 
respondence. Internet interviews can also make use of things like chat rooms. 
Chat rooms allow messages to be sent back and forth, in turn, as in a conversa- 
tion. To this extent, chat rooms provide synchronous communication in ‘real 
time’. Connection is made through an Internet service provider (ISP) and 
normally involves a process of registering when first joining. On entering a 
chat room a newcomer will normally wait and watch from the sidelines as the 
participants exchange messages. This is known as ‘lurking’. It allows the new- 
comer to get a feel for the discussions that are going on before joining in with 
something relevant and constructive. Bear in mind the fact that any comment 
that is written to a chat room becomes visible to anyone (across the world) 
who happens to be visiting the chat room at that time. Any of these can reply 
to you. The reply, again, is public. However, chat rooms also allow individuals 
to communicate with one another on a one-to-one basis where others will not 
see the messages. This is known as ‘whispering’. 

A form of interviewing can also be conducted using mailing lists. Mailing 
lists operate through an automated email program that, when it receives a 


INTERVIEWS 191 


message, simply forwards that message on to the email addresses of all the 
subscribers to the list. The mailing list contains email addresses for those who 
share an interest in specific topics or items. Some lists are moderated, some are 
not. And some list servers keep on file all the messages that are posted — thus 
being able to trace, and deter, the sending of messages that are rude, offensive 
or inappropriate for the specific list. 

Bulletin boards or newsgroups have become a readily accessible resource for 
researchers, who can identify, and make contact with, special interest groups 
on just about any imaginable topic. Subscribing to a newsgroup is much like 
putting yourself on a mailing list to receive news and information about a 
particular item of special interest. When subscribers log on to a particular group 
they automatically get sent the messages that have been posted by other mem- 
bers. Although they appear much like email, newsgroups rely on sites main- 
tained on a server computer, with different software being used to make the 
system operate. To add a contribution to the newsgroup, users send a message 
much as they would on email. And, as with email, the communication is ‘asyn- 
chronous’ -notin ‘real time’. Some newsgroups are moderated — others are not. 

From the researcher’s point of view, the use of bulletin boards or newsgroups 
has some distinct benefits: 


e The responses do not depend on who happens to be visiting the site at any 
one specific moment (an advantage over the use of chat rooms). 

e The responses tend to come from those with a strong interest in the topic. 
Newsgroups tend to be information-based forums, compared with the more 
social nature of chat rooms, and have a more serious side to them that 
means members might be better disposed to assisting researchers. 

e The responses are likely to reflect a ‘considered opinion’ rather than an 
instant ‘off-the-cuff’ remark, since the respondent has time to reflect on 
the issue. 

e It is possible to trace archive material that records past discussions — often 
over several years. 


There is, however, a significant component of the conventional interview 
that is missing when using emails, chat rooms, mailing lists, bulletin boards 
or newsgroups. There is a loss of visual clues. The interviewer and the inter- 
viewee cannot see one another. There can also be a considerable time gap 
between the question and the answer - more than would normally be 
associated with a face-to-face interview. At one level, these two features of 
Internet interviews can be regarded as an impediment to the collection of 
data. The absence of visual clues prevents the interviewer from picking up 
on important facial expressions and other non-verbal communication that 
could be valuable for understanding the interviewee’s thoughts. The time 
lapse can stultify the flow of interaction, depriving the interview of its natural 
qualities. 

The other side of the coin, as advocates of Internet interviewing point out, 
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is that these features of Internet interviews can reduce the interviewer effect. 
They can: 


e Reduce the culture and gender effects of interaction. The absence of visual clues 
means that what is stated by the interviewees is less likely to be affected by 
status factors linked to the respective age, sex, ethnicity, accent and social 
class of the interviewer and the interviewee. It acts as an ‘equalizer’ in terms 
of the communication. 

e Overcome embarrassment on some topics. The lack of face-to-face contact helps 
to make the interviewee less uncomfortable when responding to questions 
on sensitive topics. The contact, being less ‘personal’, is likely to be less 
embarrassing when covering issues where factors like the interviewer’s sex 
and age might otherwise affect the response. 

e Allow interviewees time for reflection on and consideration of the question. 
The quality of their answers might be improved as a result of them having 
time to think through the issues. 


Advantages of interviews 


e Depth of information. Interviews are particularly good at producing data 
which deal with topics in depth and in detail. Subjects can be probed, issues 
pursued and lines of investigation followed over a relatively lengthy period. 

e Insights. The researcher is likely to gain valuable insights based on the depth 
of the information gathered and the wisdom of ‘key informants’. 

e Equipment. Interviews require only simple equipment and build on conver- 
sation skills which researchers already have. 

e Informants’ priorities. Interviews are a good method for producing data 
based on informants’ priorities, opinions and ideas. Informants have the 
opportunity to expand their ideas, explain their views and identify what 
they regard as the crucial factors. 

e Flexibility. As a method for data collection, interviews are probably the most 
flexible. Adjustments to the lines of enquiry can be made during the inter- 
view itself. Interviewing allows for a developing line of enquiry. 

e High response rate. Interviews are generally prearranged and scheduled for a 
convenient time and location. This ensures a relatively high response rate. 

e Validity. Direct contact at the point of the interview means that data can be 
checked for accuracy and relevance as they are collected. In the case of 
Internet interviews, there is also the elimination of errors at the data entry 
stage. The researcher is no longer faced with passages in a tape-recorded 
interview where voices cannot be heard clearly and where there is con- 
sequently some doubt about what was actually said. Using the Internet, 
interview responses come in the form of text — text constructed directly by 
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the interviewee. This eliminates inaccuracies in the data arising from the 
process of transcription. 

e Therapeutic. Interviews can be a rewarding experience for the informant. 
Compared with questionnaires, observation and experiments, there is a 
more personal element to the method, and people tend to enjoy the rather 
rare chance to talk about their ideas at length to a person whose purpose 
is to listen and note the ideas without being critical. 


Disadvantages of interviews 


e Time-consuming. Analysis of data can be difficult and time-consuming. Data 
preparation and analysis are ‘end-loaded’ compared with, for instance, 
questionnaires which are pre-coded and where data are ready for analysis 
once they have been collected. The transcribing and coding of interview 
data are a major task for the researcher which occurs after the data have 
been collected. 

e Data analysis. The interview method tends to produce non-standard responses. 
Semi-structured and unstructured interviews produce data that are not 
pre-coded and have a relatively open format. 

e Reliability. The impact of the interviewer and of the context means that 
consistency and objectivity are hard to achieve. The data collected are, to an 
extent, unique owing to the specific context and the specific individuals 
involved. This has an adverse effect on reliability. 

e Interviewer effect. The data from interviews are based on what people say 
rather than what they do. The two may not tally. What people say they do, 
what they say they prefer and what they say they think cannot automatic- 
ally be assumed to reflect the truth. In particular, interviewee statements 
can be affected by the identity of the researcher. Internet interviews, with 
their lack of visual and verbal clues, can go a long way towards avoiding this 
problem. The absence of visual and verbal clues, some researchers argue, 
democratizes the research process by equalizing the status of researcher and 
respondent, lowering the status differentials linked with sex, age, appear- 
ance and accent, and giving the participant greater control over the process 
of data collection. The lack of such clues, equally, can help to overcome 
embarrassment on topics and allow the respondent to open up in a way that 
is unlikely to happen in the physical presence of a researcher. 

e Inhibitions. In the case of face-to-face interviews, the audio recorder (or 
video recorder) can inhibit the informant. Although the impact of the 
recording device tends to wear off quite quickly, this is not always the case. 
The interview is an artificial situation (as, of course, are experiments) where 
people are speaking for the record and on the record, and this can be daunt- 
ing for certain people. 
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e Invasion of privacy. Tactless interviewing can be an invasion of privacy and/ 
or upsetting for the informant. While interviews can be enjoyable, the other 
side of the coin is that the personal element of being interviewed carries its 
own kinds of dangers as well. 

e Resources. With face-to-face interviews the costs of interviewer’s time and 
travel can be relatively high, particularly if the informants are geographic- 
ally dispersed. 
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Checklist for the use of interviews 


When conducting interviews you should feel confident about answering 
‘yes’ to the following questions: 


Is it clear which type of interview is being used and why 
(structured, semi-structured or unstructured)? 


Is it clear whether the research is based on one-to-one 
interviews, group interviews or focus groups? 


Have field notes been written to provide supplementary 
information about the interaction during the interview? 


Have relevant details been collected about the context 
within which the interviews took place (location, prior events, 
ambience, etc.)? 


Has consideration been given to the effect of the recording 
equipment on the openness with which informants replied? 


During the interviews was the discussion monitored 
appropriately in terms of: 


informants’ key points? 

reading between the lines of what was said? 
trying to identify any inconsistencies? 

being aware of the ‘fob-off’ answer? 

looking for boastful or exaggerated answers? 
looking for answers intended simply to please 
the interviewer? 


During the interviews were prompts, probes and checks 
used to gain worthwhile, detailed insights? 


Has consideration been given to the way the researcher’s 
self-identity might have affected: 


e the interaction during the interview? 
e the interpretation of the data? 


In the case of interviews conducted via the Internet, has 
consideration been given to the impact of Internet research in 
terms of: 


e the loss of visual clues during the interaction? 
e the use of non-real-time communication? 


© M. Denscombe, The Good Research Guide. Open University Press. 
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OBSERVATION 


What is observational research? e Perception and observation 

e Systematic observation and observation schedules e Creating an 
observation schedule e Retaining the naturalness of the setting 

e Advantages of systematic observation e Disadvantages of systematic 
observation e Checklist for the use of observation schedules 


e Participant observation e What to observe, what to record e Self, 
identity and participant observation e Advantages of participant 
observation e Disadvantages of participant observation e Checklist 
for participant observation 


What is observational research? 


Observation offers the social researcher a distinct way of collecting data. It 
does not rely on what people say they do, or what they say they think. It is 
more direct than that. Instead, it draws on the direct evidence of the eye to 
witness events at first hand. It is based on the premise that, for certain pur- 
poses, it is best to observe what actually happens. 

There are essentially two kinds of observation research used in the social 
sciences. The first of these is systematic observation. Systematic observation has 
its origins in social psychology - in particular, the study of interaction in set- 
tings such as school classrooms. It is normally linked with the production of 
quantitative data and the use of statistical analysis. The second is participant 
observation. This is mainly associated with sociology and anthropology, and 
is used by researchers to infiltrate situations, sometimes as an undercover 
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operation, to understand the culture and processes of the groups being 
investigated. It is normally associated with qualitative data. 

These two methods might seem poles apart in terms of their origins and 
their use in current social research, but they share some vital characteristics: 


e Direct observation. The obvious connection is that they both rely on direct 
observation. In this respect they stand together, in contrast to methods such 
as questionnaires and interviews, which base their data on what informants 
tell the researcher, and in contrast to documents where the researcher tends 
to be one step removed from the action. 

e Fieldwork. The second common factor is their dedication to collecting data 
in real-life situations — out there in the field. In their distinct ways, they 
both involve fieldwork. The dedication to fieldwork immediately identifies 
observation as an empirical method for data collection. As a method, it 
requires the researcher to go in search of information, at first hand, rather 
than relying on secondary sources. 

e Natural settings. Fieldwork observation - distinct from laboratory observa- 
tions — occurs in situations which would have occurred whether or not the 
research had taken place. The whole point is to observe things as they nor- 
mally happen, rather than as they happen under artificially created condi- 
tions such as laboratory experiments. There is a major concern to avoid 
disrupting the naturalness of the setting when undertaking the research. In 
this approach to social research, it becomes very important to minimize the 
extent to which the presence of the researcher might alter the situation 
being researched. 

e The issue of perception. Systematic observation and participant observation 
both recognize that the process of observing is far from straightforward. 
Both are acutely sensitive to the possibility that researchers’ perceptions of 
situations might be influenced by personal factors and that the data col- 
lected could thus be unreliable. They tend to offer very different ways of 
overcoming this, but both see it as a problem that needs to be addressed. 


Perception and observation 


Two researchers looking at the same event ought to have recorded precisely the 
same things. Or should they? Using common sense, it might seem fairly obvi- 
ous that, as long as both researchers were present and able to get a good vantage 
point to see all that was happening, the records of the events — the data - should 
be identical. Yet in practice this might not be the case. It is possible that the two 
researchers will produce different records of the thing they jointly witnessed. 
Why should this be the case? Obviously, the competence of each individual 
researcher is a factor which has to be taken into consideration. The powers of 
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observation, the powers of recall and the level of commitment of individual 
researchers will vary, and this will have an effect on the observational data that 
are produced. 

The variation in records also reflects psychological factors connected to 
memory and perception. Obviously, the research information on this area is 
vast but, as far as the use of observation as a research method is concerned, 
there are three things which are particularly important that emerge from the 
work of psychologists on these topics. 

First, they point to the frailties of human memory and the way that 
we cannot possibly remember each and every detail of the events and situ- 
ations we observe. Basically, we forget most of what we see. But what we for- 
get and what we recall are not decided at random. There is a pattern to the 
way the mind manages to recall certain things and forget others. There is 
selective recall. 

Second, they point to the way the mind filters the information it receives 
through the senses. It not only acts to reduce the amount of information, it 
also operates certain ‘filters’ which let some kinds of information through to 
be experienced as ‘what happened’, while simultaneously putting up barriers 
to many others. There is selective perception. 

Third, they point to experiments which show how these filters not only let 
in some information while excluding the rest, but also boost our sensitivity to 
certain signals depending on our emotional and physical state, and our past 
experiences. What we experience can be influenced to some extent by whether 
we are, for instance, very hungry, angry, anxious, frustrated, prejudiced, etc. 
What we experience is shaped by our feelings at the moment and by the 
emotional baggage we carry around with us as a result of significant things that 
have happened to us during our lifetime. These things account for accentuated 
perception. 

The selection and organization of stimuli, then, are far from random. In fact, 
there is a tendency to highlight some information and reject some other, 
depending on: 


e Familiarity. We tend to see what we are used to seeing. If there is any ambigu- 
ity in what is being observed, we tend to interpret things according to 
frequent past experiences. 

e Past experiences. Past experience ‘teaches’ us to filter out certain ‘nasty’ 
stimuli (avoidance learning) or exaggerate desirable things. 

e Current state. Physical and emotional states can affect what is perceived by 
researchers. Physiological states such as hunger and thirst can influence the 
way we interpret what we ‘see’. Emotions, anxieties and current priorities 
can likewise alter our perceptions. 


Without delving too deeply into the psychology of perception, it is easy to 
appreciate that, as human beings, researchers do not simply observe and record 
the events they witness in some mechanical and straightforward fashion. 
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Evidence in relation to memory and perception indicates that the mind acts as 
an intermediary between ‘the world out there’ and the way it is experienced 
by the individual. There is almost inevitably an element of interpretation. 


Systematic observation and observation schedules 


The psychology of memory and perception explains why the facts recorded by 
one researcher are very likely to differ from those recorded by another, and 
why different observers can produce different impressions of the situation. 
However, all this is rather worrying when it comes to the use of observation as 
a method for collecting data. It suggests that the data are liable to be inconsis- 
tent between researchers — too dependent upon the individual and the per- 
sonal circumstances of each researcher. It implies that different observers will 
produce different data. 

It is precisely this problem which is addressed by systematic observation and 
its use of an observation schedule. The whole purpose of the schedule is to 
minimize, possibly eliminate, the variations that will arise from data based on 
individual perceptions of events and situations. Its aim is to provide a frame- 
work for observation which all observers will use, and which will enable them 
to do the following: 


e bealert to the same activities and be looking out for the same things; 

e record data systematically and thoroughly; 

e produce data which are consistent between observers, with two or more 
researchers who witness the same event recording the same data. 


To achieve these three aims, observation schedules contain a list of items that 
operate something like a checklist. The researcher who uses an observation 
schedule will monitor the items contained in the checklist and make a record 
of them as they occur. All observers will have their attention directed to the 
same things. The process of systematic observation then becomes a matter of 
measuring and recording how many times an event occurs, or how long some 
event continues. In this way, there will be a permanent record of the events 
which should be consistent between any researchers who use the schedule, 
because what is being observed is dictated by the items contained in the 
schedule. When researchers are properly trained and experienced, there 
should be what is called high ‘inter-observer’ reliability. 

The value of findings from the use of an observation schedule will depend, 
however, on how appropriate the items contained in the schedule are for 
the situation. Precise measurements of something that is irrelevant will not 
advance the research at all. It is imperative, for this reason, that the items on 
the schedule are carefully selected. The findings will only be worth something 
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if the items can be shown to be appropriate for the issues being investigated, 
and for the method of observation as well. 


Creating an observation schedule 


Literature review 


Initially, the possible features of the situation which might be observed using a 
schedule can be identified on the basis of a literature review. Such a literature 
review will present certain things as worthy of inclusion, and should allow 
the researcher to prioritize those aspects of the situation to be observed. It 
would be nice to have a huge number of items in the schedule, but this is not 
practical. Researchers are limited by the speed and accuracy with which it is 
possible to observe and record events they witness. So the items for inclusion 
need to be restricted to just the most significant and most relevant, because 
it is simply not feasible to include everything. Previous research and previous 
theories provide the key to deciding which features of the situation warrant 
the focus of attention. 


Types of events and behaviour to be recorded 


Observers can measure what happens in a variety of ways. The choice will 
depend on the events themselves and, of course, the purpose to which the 
results will be put. Observations can be based on: 


e Frequency of events. A count of the frequency with which the categories/items 
on the observation schedule occur. 

e Events at a given point in time. At given intervals (for instance, 25 seconds) 
the observer logs what is happening at that instant. This might involve 
logging numerous things which happen simultaneously at that point. 

e Duration of events. When instances occur, they are timed, so that the 
researcher gets information on the total time for each category, and when 
the categories occurred during the overall time-block for the period of 
observation. 

e Sample of people. Individuals can be observed for predetermined periods 
of time, after which the observer's attention is switched to another person 
in a rota designed to give representative data on all those involved in the 
situation. 


Suitability for observation 


When the items for inclusion in the schedule are being selected, there are 
seven conditions that need to be met. The things to be observed need to be: 
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e Overt. First and foremost, items should entail overt behaviour which is 
observable and measurable in a direct manner. Things like attitudes and 
thoughts need to be inferred by the researcher, and are not observable in a 
direct manner. 

e Obvious. They should require a minimum of interpretation by the researcher. 
The researcher should have little need to decipher the action or fathom out 
whether an action fits one or another category. 

e Context independent. Following from the point above, this means that the 
context of the situation should not have a significant impact on how 
the behaviour is to be interpreted. 

e Relevant. They should be the most relevant indications of the thing to 
be investigated. It is important that the researcher chooses only valid 
indicators, things that are a good reflection of the things being studied. 

¢ Complete. They should cover all possibilities. Care needs to be taken to 
ensure, as far as is possible, that the categories on the observation schedule 
cover the full range of possibilities and that there are not gaps which will 
become glaringly evident once the observation schedule is used in the field. 

e Precise. There should be no ambiguity about the categories. They need to 
be defined precisely and there should be no overlap between them. There 
should be the most relevant indicators of the thing being investigated. 

e Easy to record. They should occur with sufficient regularity and sequence for the 
observer to be able to log the occurrences accurately and fruitfully. If the 
category is something that is relatively rare, it will prove frustrating and 
wasteful of time to have a researcher — pen poised — waiting, waiting, wait- 
ing for something to happen. And if, like buses, the events then all come at 
once, the observer might well find it impossible to log all instances. There 
is a practical consideration here which affects the categories to be observed. 
(Choose one-at-a-time events, avoid simultaneously occurring events.) 


Sampling and observation 


When deciding what thing is to be observed, the researcher also needs to make 
a strategic decision concerning the kind of sampling to be used. Researchers 
using systematic observation generally organize their research around set 
time-blocks of observation in the field. For example, these might be one-hour 
chunks of time in situ. These time-blocks themselves need to be chosen so as to 
avoid any bias and to incorporate a representative sample of the thing in 
question. So, if the research were to be observations of interaction in school 
classrooms, the researcher would need to ensure that the research occurred 
across the full school week, the full school day and a cross-section of subjects. 
To confine observations to Friday afternoons, or to one subject such as history, 
would not provide an accurate picture across the board. 

The same applies to the selection of people for inclusion in the study. To get a 
representative picture of the event or situation, the use of systematic observa- 
tion can involve a deliberate selection of people to be observed, so that there is 
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a cross-section of the whole research population. In the case of observation in 
a school classroom, for example, the researcher could identify in advance a 
sample according to the sex and ability of students, thus ensuring that the 
observations that take place are based on a representative sample. 


Good practice: sampling of events 


Care should be taken to select the events or behaviour to be observed in accord 
with the principles of sampling (described in Chapter 1). 


Example of an observation schedule 


For the purposes of illustration, consider an observation schedule intended for 
use in art classes in a secondary school. The art classes are the ‘situation’ 
for which the observation schedule needs to be designed. Its ‘purpose’ is to 
measure the amount of lesson time wasted by students queuing to clean their 
paint brushes in the sink. Its aim might be to provide quantitative, objective 
data in support of the art teacher’s bid for resources to have a second sink 
installed in the art classroom. The simple observation schedule to be used in 
this context could take the following format. 


Location: School A 
Date: 28 April 
Time: 11 a.m. to 12 noon 


Identity of student Student starts Student arrives Queuing time 
queuing at sink 

Student Ann POLES) iL Ws) (0) 
Student Tom WAG) 11.18 3 
Student David aL UG) 11.20 5 
Student Diane 11.16 11.23 7 
Student Tony 11.17 11.24 7 
Student Eileen ial, Al's) 11.26 7 
Student 

Student 

Student 


In this example, a decision has been made to record the duration of the 
event: queuing time. It would have been possible to record the number of 
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occasions that students in the class queued, or to have noted at intervals of, 
say, 30 seconds over a one-hour period how many students were queuing at 
that moment. These would have provided slightly different kinds of results. If 
we were concerned with how queuing interrupted the concentration of stu- 
dents on a task, it would have been more appropriate to record the frequency. 
Had the aim been to look at bottlenecks in the queuing, time sampling would 
have allowed the ebb and flow of students to the sink to be shown quite clearly. 
When the aim is to support the claim that students’ time is wasted in queues 
for the sink, it is appropriate to record the total time spent in the queue. 

The item in this example is suitably straightforward to observe. We would 
presume that standing in line is an obvious and observable form of behaviour 
and that, despite some occasions when students might not be solely con- 
cerned with getting their paint brushes clean when they join the queue (they 
might be socializing or wasting time deliberately), standing in line offers a 
fairly valid indicator of the thing that is of interest to the researcher: time 
wasted queuing. 


Recording contextual factors 


Precisely because the use of an observation schedule has the tendency to 
decontextualize the things it records, more advanced practice in this area has 
made a point of insisting that researchers collect information about relevant 
background matters whenever they use a schedule (Galton et al. 1980). Such 
background information helps to explain the events observed, and should 
be logged with the schedule results to help the observer understand the data 
he or she has collected. 


Good practice: using field notes to complement systematic observation 


The quantitative data produced by systematic observation schedules should 
be complemented by field notes (qualitative data) in which the researcher 
(1) describes the context and (2) records his/her impressions about the 
circumstances surrounding the events or behaviour being observed. 


Retaining the naturalness of the setting 


With systematic observation, the issue of retaining the naturalness of the set- 
ting hinges on the prospect of the researcher fading into the background and 
becoming, to all intents and purposes, invisible. At first this might seem an 
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implausible thing. Armed with a clipboard and pen, and looking like a ‘time 
and motion’ researcher, it would seem unlikely that such systematic observa- 
tion could avoid disrupting the events it seeks to measure. However, those who 
engage in this style of research report that it is indeed possible to ‘merge into 
the wallpaper’ and have no discernible impact. They stress that to minimize the 
likelihood of disruption researchers should pay attention to three things: 


Link up with the Observer effect, pp. 69, 142-3, 178-80, 326 


Positioning. Unobtrusive positioning is vital. But the researcher still needs to 
be able to view the whole arena of action. 

Avoiding interaction. The advice here is to be ‘socially invisible’, not engaging 
with the participants in the setting if at all possible. 

Time on site. The experience of systematic observers assures them that the 
longer they are ‘on site’, the more their presence is taken for granted and the 
less they have any significant effect on proceedings. 


a 
o 


Advantages of systematic observation 


Direct data collection. It directly records what people do, as distinct from what 
they say they do. 

Systematic and rigorous. The use of an observation schedule provides an ans- 
wer to the problems associated with the selective perception of observers, 
and it appears to produce objective observations. The schedule effectively 
eliminates any bias from the current emotions or personal background of 
the observer. 

Efficient. It provides a means for collecting substantial amounts of data in a 
relatively short timespan. 

Pre-coded data. It produces quantitative data which are pre-coded and ready 
for analysis. 

Reliability. When properly established, it should achieve high levels of 
interobserver reliability in the sense that two or more observers using a 
schedule should record very similar data. 


Disadvantages of systematic observation 


Behaviour, not intentions. Its focus on overt behaviour describes what hap- 
pens, but not why it happens. It does not deal with the intentions that 
motivated the behaviour. 
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e Oversimplifies. It assumes that overt behaviours can be measured in terms of 
categories that are fairly straightforward and unproblematic. This is prem- 
ised on the idea that the observer and the observed share an understanding 
of the overt behaviour, and that the behaviour has no double meaning, 
hidden meaning or confusion associated with it. As such, systematic obser- 
vation has the in-built potential to oversimplify; to ignore or distort the 
subtleties of the situation. 

¢ Contextual information. Observation schedules, by themselves, tend to miss 
contextual information which has a bearing on the behaviours recorded. It 
is not a holistic approach. 

e Naturalness of the setting. Despite the confidence arising from experience, 
there remains a question mark about the observer’s ability to fade into the 
background. Can a researcher with a clipboard and observation schedule 
really avoid disrupting the naturalness of the setting? 


Checklist for the use of observation schedules 


When using an observation schedule you should feel confident about 
answering ‘yes’ to the following questions: 


Has the observation schedule been piloted? 


Have efforts been made to minimize any disturbance to the 
naturalness of the setting caused by the presence of the observer? 
Do the planned periods for observation provide a representative 
sample (time, place, context)? 
Are the events/behaviour to be observed: 
e sufficiently clear-cut and unambiguous to 
allow reliable coding? 
e the most relevant indicators for the purposes 
of the research? 
Is the schedule complete (incorporating all likely 
categories of events/behaviour)? 
Do the events/behaviour occur regularly enough to 
provide sufficient data? 
Does the schedule avoid multiple simultaneous occurrences 
of the event/behaviour which might prevent accurate coding? 
Is the kind of sampling (event/point/time) the most appropriate? 


Is there provision for the collection of contextual information to 
accompany the schedule data? 


© M. Denscombe, The Good Research Guide. Open University Press. 
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Participant observation 


A classic definition of participant observation spells out the crucial character- 
istics of this approach, and the things which distinguish it from systematic 
observation: 


By participant observation we mean the method in which the observer 
participates in the daily life of the people under study, either openly in the 
role of researcher or covertly in some disguised role, observing things that 
happen, listening to what is said, and questioning people, over some 
length of time. 

(Becker and Geer 1957: 28) 


As Becker and Geer indicate, the participant observer can operate in a com- 
pletely covert fashion — like an undercover agent whose success depends on 
remaining undetected, whose purpose remains top secret. If no one knows 
about the research except the researcher, the logic is that no one will act in 
anything but a normal way. Preserving the naturalness of the setting is the key 
priority for participant observation. The principal concern is to minimize 
disruption so as to be able to see things as they normally occur — unaffected 
by any awareness that research is happening. 

Another priority is to gain information about cultures or events which 
would remain hidden from view if the researcher were to adopt other methods. 
Such information could remain hidden for two reasons. Those involved in the 
culture or event could deliberately hide or disguise certain ‘truths’ on occasions 
when they are ‘under the microscope’. In this case, covert participant observa- 
tion reveals such events by doing the research secretly. Nothing will get 
hidden. The ‘participant observer’ will be able to see everything - the real 
happenings, warts and all. Alternatively, aspects of the culture/events could 
remain hidden because researchers using other methods would remain 
unaware of them. In this case, participant observation discloses things through 
the researcher’s experience of participating in the culture or event. Only by 
experiencing things from the insider’s point of view does the researcher become 
aware of the crucial factors explaining the culture or event. With participant 
observation the aim is to get insights into cultures and events — insights only 
coming to one who experiences things as an insider. 

The nature of participant observation also allows the researcher to place 
greater emphasis on depth rather than breadth of data. In principle, participant 
observation can produce data which are better able than is the case with other 
methods to reflect the detail, the subtleties, the complexity and the intercon- 
nectedness of the social world it investigates. In the spirit of anthropology, 
cultures and events are subject in the first instance to detailed study. Attention 
is given to intricate details of the social world being studied, and on the rou- 
tine as well as the special and the extraordinary. Emphasis is placed on holistic 
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understanding, in which the individual things being studied are examined in 
terms of their relationships with other parts, and with the whole event or 
culture. And, in similar vein, things are examined in relation to their context. 
In those respects, participant observation scores highly in terms of the validity 
of the data. 


2 


(J PAd 


Link up with Ethnography, Chapter 4 


However, the participant observer role need not involve this total immersion. 
There are versions of participant observation in which the participation elem- 
ent is rather different. Participation, in this sense, means ‘being there’ and ‘in 
the middle of the action’. One possibility here is that the researcher’s role as 
an observer is still kept secret. Particularly when contemplating fieldwork in 
more salacious settings, this strategy can be very valuable (Humphreys 1970). 
O’Connell Davidson (1995), studying the working practices of a prostitute, 
acted occasionally as a receptionist for Madame Desirée, thus allowing her to 
be part of the normal scene but also allowing a judicious distance from the 
heart of the action. 

Another possibility involves hanging out with a group rather than becoming 
a member of that group. And this can allow the researcher to be open about 
their purpose — to get consent for the research — in a way that is denied to 
the total version of participant observation. Of course, the downside of this 
is that the presence of the researcher can serve to disrupt the naturalness 
of the setting. 

There are numerous variations which have been used that tinker with the 
extent of total participation and the extent of open observation, but the essen- 
tial notion of participant observation revolves around the three possibilities: 


e Total participation, where the researcher's role is kept secret. The researcher 
assumes the role of someone who normally participates in the setting. Con- 
sent cannot be gained for the research, which poses ethical problems. 

e Participation in the normal setting, where the researcher’s role may be known 
to certain ‘gatekeepers’, but may be hidden from most of those in the set- 
ting. The role adopted in this type of participant observation is chosen 
deliberately to permit observation without affecting the naturalness of the 
setting, but it also allows the researcher to keep a distance from the key 
group under study. This distance might be warranted on the grounds of 
propriety, or the researcher lacks the personal credentials to take on the role 
in question. 

e Participation as observer, where the researcher’s identity as a researcher is 
openly recognized — thus having the advantages of gaining informed con- 
sent from those involved — and takes the form of ‘shadowing’ a person or 
group through normal life, witnessing at first hand and in intimate detail 
the culture/events of interest. 
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What to observe, what to record 


Starting fieldwork 


The researcher should not enter the field with pre-established hypotheses to 
be tested. The researcher is there to learn about the situation. The longer the 
researcher is able to spend ‘on site’ the better, because the longer he or she is 
part of the action, the more can be learnt about the situation. Good participant 
observation demands that the researcher devotes considerable time to the 
fieldwork. This is not a hit-and-run research method. Time on site is needed 
to gain trust, to establish rapport and foster insights, insights that are the 
trademark of participant observation as a research method. 

Then there is the question of what to observe during the time on site. The 
researcher should start out being fairly non-selective in terms of what he or she 
observes. Before anything else, the participant observer should aim to get an 
‘overall feel’ for the situation, and to do this he or she should engage in what 
can be termed ‘holistic observation’. 

Of course, getting a general feel for the setting, while it is valuable as a 
background scene-setting device, is really a prelude to more focused observa- 
tions. As things emerge which appear to have particular significance or interest, 
observation will shift from the broad canvas of activity in the setting towards 
specific areas. Things which emerge as important, strange or unusual invite 
closer scrutiny. 

Following from focused observations, the researcher might be able to under- 
take special observations which concentrate on aspects of the setting in which 
there appear to be things which are unexpected or contradictory. Attention 
can be focused upon things that, according to the observer’s common sense, 
ought not to happen. 

Finally, observations can try to identify issues and problems which partici- 
pants themselves regard as crucial. The point is to observe instances which 
indicate how members of the setting see things — their views, beliefs and 
experiences. 


Making field notes 


The fieldwork researcher needs to translate the observations into some per- 
manent record at the very earliest opportunity. This might be ‘field notes’ in 
the form of written records or tape-recorded memos. Whatever the form, the 
researcher doing fieldwork needs to develop a strategy for writing up field 
notes as soon as possible after the observation. 

The need to do so stems from two things. First, the human memory is not 
only selective, but also frail. It is so easy to forget things, particularly the minor 
incidents and passing thoughts, if field notes are delayed for a matter of days, 
let alone weeks. Field notes are urgent business. The researcher needs to build 
into the research some provision to make the field notes on a regular and 
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prompt basis. The second factor involved here is the general need to take field 
notes outside the arena of action. To take field notes while engaging in the 
action as a participant, to state the obvious, would be (1) to disrupt the natur- 
alness of the setting, and (2) to disclose the researcher’s role as observer. As a 
general rule, then, participant observers need to establish occasions during 
fieldwork, or very soon afterwards, when they can make field notes in private 
and unknown to those being observed. The simplest strategy is to write up the 
field notes as soon as you get home - assuming that home is separate from the 
field being studied. 


Good practice: making field notes with participant observation 


Detailed field notes should be made to accompany participant observation. 
These notes should be made as soon as possible following each episode of 
observation. 


Ethics 


Participant observation can pose particular ethical problems for the researcher. 
If ‘total’ participation is used, then those being studied will not be aware of the 
research or their role in it. They can hardly give ‘informed consent’. The justi- 
fication for such covert research cannot depend on consent, but draws instead 
on two other arguments. First, if it can be demonstrated that none of those 
who were studied suffered as a result of being observed, the researcher can argue 
that certain ethical standards were maintained. Second, and linked, if the 
researcher can show that the identities of those involved were never disclosed, 
again there is a reasonable case for saying that the participant observation was 
conducted in an ethical manner. 

Whichever variant of participant observation is used, there is the possibility 
that confidential material might ‘fall into the hands’ of the researcher. Now, 
while this is true of most research methods, its prospects are exacerbated 
with the use of participant observation, owing to the closeness and intimacy of 
the researcher’s role vis-a-vis those being researched. Confidential material 
might be disclosed inadvertently by someone who does not know the research 
interest of the participant. Or, possibly even more problematic, things might 
get revealed as a result of the trust and rapport developed between the 
researcher and those being observed. This could be true for any of the variants 
of participant observation. The ethical problem is whether to use such material 
and how to use it. And here the guidelines are quite clear: (1) any use of 
the material should ensure that no one suffers as a result; and (2) any use of 
the material should avoid disclosing the identities of those involved. Any 
departure from these guidelines would need very special consideration and 
justification. 
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Self, identity and participant observation 


Equipment for research: the ‘self’ 


One of the attractions of participant observation is that it hinges on the 
researcher's ‘self’, and does not call on much by the way of technical back-up 
in the form of gadgets or software. Nor does it tend to produce data that call for 
statistical analysis. The key instrument of participant observation methods is the 
researcher as a person. 

This suggests that there is little in the way of ‘entry costs’ to act as a deter- 
rent. Equipment costs are very low. There might appear to be no need for 
training (though this, of course, would be a fallacy). The researcher, it might 
seem, can jump right into the fieldwork and get on with it. However, as we see 
in the next sections, this dependence on the ‘self’ is not altogether a straight- 
forward advantage. 


Access to settings 


Access is not necessarily a matter of getting approval from relevant authorities 
or getting a ‘gatekeeper’ to help open doors to the necessary contacts and 
settings. As well as these, when engaging in the total version of participant 
observation there is a special, peculiar issue affecting access. If the researcher 
is to adopt a role in the setting then he or she needs to have the necessary 
credentials — both personal and qualifications. 

To operate ‘under cover’ in a setting it is obvious that the researcher should 
not stand out like a sore thumb. Depending on the situation, this can effect- 
ively exclude many researchers from many roles. The age factor will bar 
most (all?) researchers from using participant observation to investigate stu- 
dent cultures in schools. Observing the setting as a teacher is a more likely 
prospect. Sex will offer other barriers. Male researchers will be hard pushed 
to use total participant observation for the study of, for example, cocktail 
waitresses. Observing as a barman in the setting is a more likely prospect. Black 
researchers will find it exceptionally difficult to infiltrate the Ku Klux Klan. 
The biological factors place severe constraints on access to situations. Skills 
and qualifications provide another barrier. To participate in the sense of 
adopting a role, it is necessary to have the necessary skills and qualifications 
associated with that group. As Polsky (1967) points out, his study of pool 
hall hustling was only possible as a participant observer because — through a 
‘misspent youth’ — he was already something of an accomplished pool player 
himself. The would-be researcher, however, might be reluctant or unable 
to achieve such a skill specifically for the purpose of a piece of research. Follow- 
ing the logic here, there are many, many roles which the researcher will 
be unable to adopt — from brain surgeon to tree surgeon — because of a lack of 
credentials. 
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Selecting a topic 


In view of the constraints on access and the potential hazards of doing field- 
work as a full participant, there are two things which emerge that have a direct 
bearing on the selection of a topic. 


e To a large extent, researchers who do participant observation have their 
topic selected for them on the basis of their pre-existing personal attributes. 
The ‘choice’ is rarely much of a free choice. The researcher’s self — age, sex, 
ethnicity, qualifications, skills, social background and lifestyle — tends to 
direct the possibilities and provide major constraints on the roles that can be 
adopted. 

e While it is arguably the most revealing and sensitive of research methods 
in the social sciences, it is also very demanding. It is not a soft option. The 
level of commitment needed for full participant observation can be far 
more than that demanded by other methods - commitment in terms of 
researcher’s time and the degree to which the act of research invades the 
routine life of the researcher. 


It is not surprising, then, that many of the fascinating studies emerge as ‘one- 
offs’ in which researchers have explored an area of social life for which they 
are uniquely qualified to participate through their own past experience. It is far 
more unusual to find examples where researchers have been deliberately 
employed to infiltrate a group (e.g. Festinger et al. 1956) or where researchers 
have consciously adopted a role which is alien to them and which involves 
danger and discomfort (e.g. Griffin 1962). 

Another consequence of the restrictions to full participation is the decision 
of many social researchers to opt for the version of participant observation 
which is not ‘total participation’. Participation in the setting and participation 
as observer offer approaches which side-step some of the dangers of total par- 
ticipation and offer a more palatable experience for the researcher on many 
occasions (e.g. Whyte [1943] 1981; Humphreys 1970; O’Connell Davidson 
1995). 


Good practice: choosing a topic to suit your ‘self’ 


Researchers using participant observation generally choose a topic about 
which they have some insider knowledge and personal experience. The setting 
is normally one where the researcher can fit in easily and comfortably without 
disturbing the naturalness of the setting. In both sense, then, the choice of 
topic tends to suit the researcher’s ‘self’. 
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Going native 


If the researcher has the necessary credentials and personal resources to gain 
access as a participant observer, they are then faced with the need to operate at 
two levels while in the setting. The success of participant observation relies on 
the researcher’s ability, at one and the same time, to be a member of the group 
being studied and to retain a certain detachment which allows for the research 
observation aspect of the role. It is vital, in this respect, that the researcher does 
not lose sight of the original purpose for being there and does not get engulfed 
by the circumstances or swallowed up. The success of participant observation 
depends on being able to walk a tightrope between the involvement and 
passion associated with full participation and the cool detachment associated 
with research observation. If the researcher’s self gets lost, this is rather like an 
anthropologist forgetting all about their research and settling down to live out 
their days as a member of the ‘tribe’ that they had originally set out to study: 
‘going native’. 


Going native is an objectionable term, deservedly, for an objectionable 
phenomenon. It means over-identifying with the respondents, and losing 
the researcher’s twin perspective of her own culture and, more import- 
antly, of her ‘research’ and outlook. 

(Delamont 1992: 34) 


Dangers of fieldwork 


Doing participant observation can be dangerous. First, there is physical danger. 
As Lee (1995) points out, being physically injured while doing fieldwork 
is fairly unlikely but, depending on the circumstances, cannot be ignored 
as a possibility. It is a potential built into some forms of fieldwork. Danger 
lurks for anthropologists who travel in remote regions with inhospitable cli- 
mates and treacherous terrains. In the early years of the century, evidently, 
there were instances where anthropologists were actually killed by the people 
they were studying (Howell 1990). Danger lurks for political scientists who 
operate in unstable societies where the rule of law is tenuous and civilians 
can get caught up in factional disputes. Danger lurks for sociologists and eth- 
nographers when they make contact with groups whose activities are on the 
margins of, or even outside, the law. As they tap into the underworlds of drugs, 
prostitution, football hooligans, bikers, religious sects and the like, they are 
taking a risk. 

Imagine, for the sake of illustration, that a researcher sees the need to 
investigate the culture surrounding the use of hard drugs by young people. 
The use of a participant observation approach would seem well suited to such 
a study. After all, the ‘reality’ of how, when and why hard drugs are used is 
hardly likely to emerge by using questionnaires or experiments. Interviews 
might be useful, but there is a prima facie case for participant observation as the 
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method best suited to this particular issue. Provided that a researcher can over- 
come the first hurdle — to ‘look the part’ — the fieldwork then involves a range 
of dangers. There is actual physical danger. This is not just a reference to the 
prospect of getting mugged or assaulted, or of retribution if the cover is blown 
at some stage. There is also the danger posed by the lifestyle itself and the 
impact on health of a changed diet and changed accommodation. Changing 
lifestyle carries its own hazards. Of course, if the researcher were to become 
dependent on the use of hard drugs, the health consequences could be far 
more dramatic. 

The fieldwork could involve a second danger: legal prosecution. Being ‘part of 
the scene’ when hard drugs are around immediately puts the researcher at risk 
of prosecution. There are no special immunities afforded to social researchers. 

The researcher who chooses to engage in such fieldwork might also jeopard- 
ize his or her social well-being. The ‘other’ life he or she is called upon to live 
for the purposes of the research can have an adverse effect on domestic life, on 
relationships with others and on commitments to do with work and leisure 
which make up the ‘normal’ life of the researcher. The researcher, in effect, needs 
to sustain two lifestyles, and these may not be compatible. Being away from 
home, being out late and doing fieldwork at ‘unsocial’ hours can tax the 
patience of the nearest and dearest. (Let alone what the researcher does while 
in the field!) 

Finally, there is the psychological danger resulting from the dual existence 
demanded of fieldwork such as this. The lifestyle, at its worst, can have some- 
thing of a traumatic effect on the researcher, or can have a lasting or perman- 
ent effect on the researcher’s personality. 


Good practice: avoid dangerous fieldwork 


A risk assessment of the fieldwork setting should be conducted and potential 
dangers identified. Although experienced researchers might venture into dan- 
gerous contexts to conduct their participant observation this should not be 
done by project researchers. 


Advantages of participant observation 


e Basic equipment. Participant observation uses the researcher’s ‘self’ as the 
main instrument of research, and therefore requires little by way of tech- 
nical/statistical support. 

e Non-interference. It stands a better chance of retaining the naturalness of the 
setting than other social research methods. 
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e Insights. It provides a good platform for gaining rich insights into social 
processes and is suited to dealing with complex realities. 

e Ecological validity. The data produced by participant observation have the 
potential to be particularly context sensitive and ecologically valid. 

e Holistic. Participant observation studies offer holistic explanations incorpo- 
rating the relationships between various factors. 

e Subjects’ points of view. As a method of social research, participant observation 
is good for getting at actors’ meanings as they see them. 


Disadvantages of participant observation 


e Access. There are limited options open to the researcher about which roles to 
adopt or settings to participate in. 

e Commitment. Participant observation can be a very demanding method in 
terms of personal commitment and personal resources. 

e Danger. Participant observation can be potentially hazardous for the 
researcher; physically, legally, socially and psychologically risky. 

e Reliability. Dependence on the ‘self’ of the researcher and on the use of field 
notes as data leads to a lack of verifiable data. Reliability is open to doubt. 
Because participant observation relies so crucially on the researcher’s ‘self’ 
as the instrument of research, it becomes exceedingly difficult to repeat a 
study to check for reliability. The dependence on field notes for data, con- 
structed (soon) after fieldwork and based on the researcher’s recollections of 
events, does little to encourage those who would want to apply conventional 
criteria for reliability to this method. 

e Representativeness of the data. There are problems of generalizing from the 
research. The focal role of the researcher’s ‘self’ and the emphasis on detailed 
research of the particular setting open participant observation to the criti- 
cism that it is difficult to generalize from the findings. In one sense, this 
might hold water as a valid criticism. After all, the situations for research 
using participant observation are not selected on the grounds of being 
representative. As we have seen, they tend to be chosen on the basis of a 
mixture of availability and convenience. However, it might be argued that it 
is inappropriate to apply standard criteria of reliability and generalizability 
to this method. 

e Deception. When researchers opt to conduct full participation, keeping their 
true identity and purpose secret from others in the setting, there are ethical 
problems arising from the absence of consent on the part of those being 
observed, and of deception by the researcher. 
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Checklist for participant observation 


When undertaking participant observation you should feel confident about 
answering ‘yes’ to the following questions: 


Has a risk-assessment been conducted which shows that 
there are no specific dangers linked with the fieldwork setting? 


Is it clear which type of participant observation was used 
(total participation, participation in normal setting, participation 
as observer)? 


Is there evidence that the participant observation did not 
disturb the naturalness of the setting? 


Has consideration been given to the ethics of the fieldwork 
(secrecy, consent, confidentiality)? 


Has the influence of the researcher’s self-identity been 
examined in terms of: 


e the choice of fieldwork situation? 
e access to the setting? 
e the perception of events and cultures? 


Was sufficient time spent in the field: 


e to allow trust and rapport to develop? 
e toallow detailed observations and an in-depth understanding 
of the situation (detail, context, interconnections)? 


Does the participant observation allow insights to events 
and meanings that would not be possible using other methods? 


Were field notes made at the time or soon after participating 
in the field? 


© M. Denscombe, The Good Research Guide. Open University Press. 
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DOCUMENTS 


Sources of documentary data e Access to documentary sources e The 
validity of documentary data e The credibility of articles, journals and 
books e The evaluation of Internet documents and web pages e The 

credibility of official statistics e The credibility of records of meetings 


e Image-based research e The use of ‘created’ images e The use of 
‘found’ images e Advantages of documentary research e Disadvantages 
of documentary research e Checklist for the use of documents (text and 
visual images) 


Sources of documentary data 


Documents can be treated as a source of data in their own right — in effect an 
alternative to questionnaires, interviews or observation. The documentary 
sources identified below are written sources. There are, though, alternative 
types of documents for research, which take the form of visual sources (pic- 
tures, artefacts) and even sounds (music). These also constitute some form of 
‘document’ which has a value for research but, because they are used relatively 
rarely within the social sciences, the initial comments in this chapter are 
restricted to written forms of documents. 
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Government publications and official statistics 


At first glance government publications and official statistics would seem to 
be an attractive proposition for the social researcher. They would appear to 
provide a documentary source of information that is: 


e Authoritative. Since the data have been produced by the state, employing 
large resources and expert professionals, they tend to have credibility. 

e Objective. Since the data have been produced by officials, they might be 
regarded as impartial. 

e Factual. In the case of the statistics, they take the form of numbers that 
are amenable to computer storage/analysis, and constitute ‘hard facts’ over 
which there can be no ambiguity. 


It is not surprising, then, that in the Western world government publications 
and official statistics have come to provide a key source of documentary 
information for social scientists. However, the extent to which such documents 
can live up to the image of being authoritative, objective and factual depends 
very much on the data they contain (see below). 


Newspapers and magazines 


The ‘press’ provides a potentially valuable source of information for research 
purposes. One reason for this is that newspapers and magazines can supply 
good, up-to-date information. In this case, the value of the newspaper or 
magazine for the research will stem from one or a combination of: 


e the expertise of the journalists; 
e the specialism of the publication; 
e the insider information which the correspondents can uncover. 


So, for example, in the UK, business researchers might use The Economist or 
the Financial Times for these reasons. Of course, the discerning researcher 
will also realize that there are plenty of newspapers and magazines whose con- 
tents should not be relied upon to reflect anything approaching an objective 
account of real events! 


Records of meetings 


The bureaucratization of industrial society has created a wealth of documen- 
tation in relation to administration, policy, management, finance and com- 
merce. These provide an abundant source of data for social researchers in 
whatever field they operate. 

The purpose of most such documentation is to enhance accountability. The 
records which are kept of meetings (minutes), the records kept of transactions, 
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the records kept of finances, etc. are kept in principle so that people and 
institutions can be held accountable for their actions. This means that the 
records need to have two qualities, both of which happen to be of particular 
value for research. 


1 They need to contain a fairly systematic picture of things that have happened. 
These might be decisions of a committee or transfers of money between 
accounts. Whatever the records, though, the principle behind them is that 
they provide a detailed and accurate picture of what took place. They have 
got to make events sufficiently transparent for the readers to comprehend 
what took place and why. 

2 They should be publicly available. The records only serve the function of 
accountability to the extent that they are made available to relevant people 
to scrutinize. 


Letters and memos 


Private correspondence between people can be used for research purposes. 
This can take the form of memos sent between people at work or even personal 
letters exchanged between people. The more private the correspondence, 
of course, the more difficult it is for the researcher to gain access to the 
documents. 

These are really at the other end of the spectrum from the publicly available 
reports, and pose far more of a challenge for the researcher when it comes to 
getting hold of such documents and, especially, when it comes to getting 
permission to use them as research data. 

Letters and memos also differ from reports in terms of the extent to which 
there is any formal obligation on the writer to give a full and accurate portrayal 
of events. Because they are written to specific people, rather than for a broader 
public, their contents are likely to rely far more on assumptions about what 
the other person already knows or what that person feels. They are more likely 
to ‘fill in the bits’ rather than paint the whole picture. They can be expected to 
be from a personal point of view rather than be impartial. When contemplat- 
ing the use of letters or memos as data for research, then, you need to recognize 
that they are not very reliable as accounts which depict objective reality, but 
they are extremely valuable as a source which reveals the writer’s own percep- 
tions and views of events. 


Diaries 


As a source of documentary data, diaries are written by people whose thoughts 
and behaviour the researcher wishes to study. For research purposes such 
diaries are important in terms of recording things that have already hap- 
pened. We are not talking about the kind of diaries which act as a planner, 
noting commitments in the future that need to be scheduled. For research 
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purposes, the diary is normally a retrospective account of things that have 
happened. 

There are three crucial elements to this kind of diary — three elements which, 
incidentally, are shared with the literary diary and the statesman’s diary: 


e factual data: a log of things that happened, decisions made and people 
involved; 

e significant incidents: the identification of things seen as particularly import- 
ant and a description of the diary-writer’s priorities; 

e personal interpretation: a personal reflection and interpretation of happen- 
ings, plus an account of the personal feelings and emotions surrounding the 
events described. 


Each of these three things has the potential to provide a rich source of data for 
the researcher. However, the accounts they provide should not be used by 
the researcher as a statement of objective fact. That would be very naive. As a 
retrospective account, diaries must always be seen as a version of things as seen 
by the writer, filtered through the writer’s past experiences, own identity, own 
aspirations and own personality. 

It is also worth noting that diaries, as a form of documentary data, lend 
themselves to being analysed in a variety of ways. As a method of data collec- 
tion they do not prescribe such matters and, as Alaszewski (2006) argues, they 
can be analysed using approaches as widely different as content analysis, 
grounded theory and discourse analysis. 


Link up with Qualitative data, Chapter 14 A 
"ee 
Website pages and the Internet 


Documents, as a form of data, include material obtained via the Internet. In a 
sense, the medium through which the document is obtained is not the issue. 
We can read newspapers in their original paper form, or we can read them 
on microfiche or via a CD-ROM. Equally, we can obtain documents through 
website pages or email, and this does not, of itself, have a bearing on the use of 
the output as a document for research. Websites, though, can be treated as 
documents in their own right. Home pages, etc. can be treated as a form of 
document, and their content analysed in terms of the text and images they 
contain. In effect, they can be treated like online documents. 
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Access to documentary sources 


Probably the greatest attraction of using documentary sources is their accessi- 
bility. To get hold of the material the researcher needs only to visit the library 
or go online via a computer. The Internet provides the researcher with access 
to a host of government sites that provide a wide range of information on 
social and economic factors. In the UK a list of government websites can be 
found at <http://www.open.gov.uk>, and in the USA there are sites like <http:/ 
/www.access.gpo.gov> and <http://www.fedworld.gov> that offer entry points 
to search for official data and government publications. This means that 
vast amounts of information are conveniently available without much cost, 
without much delay, without prior appointment, without the need for author- 
ization and without any likelihood of ethical problems. Documents, in other 
words, pose considerably fewer problems than people as a source of data for 
social researchers. 

There are times, however, when the use of documents does not completely 
side-step the kind of problems that face researchers using other sources. 
While access to documents in the public domain is certainly straightforward, 
researchers can sometimes need to use materials whose availability is delib- 
erately restricted. When this is the case, access becomes a crucial part of 
the research method - more than something that can be taken for granted. 
When documents are not available, researchers need to enter negotiations 
with those who hold the data in order to persuade them to allow the researcher 
the privilege of access to the material. Documents such as police files and 
medical records, and certain kinds of company reports and memos, might 
rightly be considered sensitive and confidential by those who ‘own’ them, and 
they are only likely to grant access when they are convinced that the 
researcher will honour the confidentiality of the material and use it in a way 
that will not harm the interests of anyone concerned. 


Good practice: gaining access to documentary sources of data 


Researchers should check in advance whether their access to documentary 
data sources will need authorization or require payment. The feasibility of 
using documentary research should then be assessed bearing in mind any 
restrictions these factors might impose on full access to the documents. 


Access to information on the Internet can be controlled by the actions of 
certain strategic ‘gatekeepers’ who can influence what information is made 
available and who is allowed access to that information. Some information, 
such as academic articles and market research data, will only be accessible on 
condition of payment of money. Separate from this, Internet researchers need 
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to be sensitive to the role of ‘moderators’ and ‘webmasters’ in filtering the 
information. Some sites, but by no means all, are controlled by people whose 
role it is to monitor the content of the site and the conduct of those who use 
it. Moderators (mailing lists, bulletin boards, newsgroups, chat rooms) and 
webmasters (email servers) acting in this capacity are effectively ‘gatekeepers’. 
They have the power to deny access to the site and might need to be persuaded 
of the relevance and worthiness of a proposed piece of research before they 
will grant access. 

In the case of documents considered to be secret, of course, the holders will 
seek to deny any access to outsiders. Companies may have secret strategic 
plans regarded as too sensitive for even a bona fide researcher. The government 
wishes to keep many documents secret ‘in the national interest’. Records of 
illegal activities, whether a drug smuggler’s diary or a second set of books for 
VAT returns, are extremely unlikely to be opened to the researcher. Access will 
not be willingly given. To get access to such secret documents requires the 
approach of investigative journalism or participant observation with a degree 
of deception and covert infiltration being used. 


Table 12.1 Access to documents for research 


Type of document 


Examples 


Access via 


Typical use 


Public domain 


Restricted access 


Secret 


Books, journals, 
official statistics, 
some company 
records 


Medical records, 
police files, internal 
memos, personal 
papers/diaries, 
some tax accounts 


Cabinet minutes, 
illegal trade, second 
set of books for VAT 
or Inland Revenue, 
corporate plans 


Libraries, Internet, 
Office for National 
Statistics 


Negotiations with 
gatekeepers, 
sponsors 


Insider knowledge, 
participation, 
deception 


The validity of documentary data 


Academic research 
and consultancy 


Consultancy and 
academic research 


Investigative 
journalism, fraud 
detection, 
undercover work 


For the purposes of research, documentary sources should never be accepted at 
face value. Their validity is something that needs to be established rather than 
being taken for granted and, as Platt (1981) and Scott (1990) have argued, 
documents need to be evaluated in relation to four basic criteria. 
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Authenticity 


Is it the genuine article? Is it the real thing? Can we be satisfied that the 
document is what it purports to be — not a fake or a forgery? 


Representativeness 


Is the document typical of its type? Does it represent a typical instance of the 
thing it portrays? Is the document complete? Has it been edited? Is the extract 
treated ‘in context’? 


Meaning 


Is the meaning of the words clear and unambiguous? Are there hidden mean- 
ings? Does the document contain argot and subtle codes? Are there meanings 
which involve ‘what’s left unsaid’ or ‘reading between the lines’? 


Credibility 


Is it accurate? Is it free from bias and errors? This will depend on factors like: 


e What purpose was the document written for? 

e Who produced the document? What was the status of the author and did he 
or she have a particular belief or persuasion that would colour the version 
of things? 

e If it reports on events, was it a first-hand report directly witnessed by the 
author? How long after the event was the document written? 

e When was the document produced? In what social context and climate? 


The credibility of articles, journals and books 


Academic journals and commercial publishers generally have their material 
refereed by experts in the field before the work is published, so the researcher 
has some assurance about the quality of their content. However, this offers no 
absolute guarantee. And how does the reader find out which journals are ref- 
ereed and which publishers are reputable? Some rules of thumb might help 
here. For journals: 


e How long has the journal existed? Generally, a journal which has existed for 
a long time will be OK. Less confidence can be held about new journals. 
However, this does not mean old equals good, new equals bad: that would 
be far too simplistic. 
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e Does the title involve a national title (e.g. British Medical Journal, American 
Journal of Sociology)? Again, there is nothing cast iron about this, but it offers 
some clue as to the standing of the journal. 

e Is the journal published by, or on behalf of, a professional association or 
some authority? 

e Does the journal contain a list of its editorial board and editorial advisers, 
and do these people strike you as ‘of high standing’ in the field? 

e Is there a clear statement that articles are refereed? 


For books and their publishers, the following questions might help with the 
evaluation of the sources. 


e Have you heard of the publisher before? If so, this might be reassuring. 

e Is the publisher a university press (e.g. Cambridge University Press)? If so, 
again this offers some confidence about the academic quality of the work 
being published. 

e Is the book in a second or subsequent edition? If so, there has been sufficient 
demand for the book to warrant new editions, and this would suggest that 
it has something worthwhile to say. The same applies to a lesser extent 
where a book has been reprinted several times. Look for the details, usually 
contained on the imprint page between the title and contents pages. 

e If itis a library book, how frequently has it been out on loan recently? The 
chances are that a book that is frequently on loan has been recommended, 
and that is why it is in demand. 


The evaluation of Internet documents and web pages 


The evaluation of the documents downloaded from the Internet centres around 
one issue more than others. Because there are few restrictions on what is 
placed on the Internet, any documents to be used for research need to be 
considered very carefully in terms of their authorship, their credibility and 
their authenticity. The documents need to be subjected to the researcher’s own 
quality audit along the lines of those recommended in relation to books and 
journals — but with even more vigour and rigour. 

When it comes to the evaluation of the web pages, a similar point applies. 
For research purposes judgements need to be made about the credibility of the 
site where the pages are located. Judgements about this can be made on the 
basis of four criteria. 


e The authoritativeness of the site. A university or government site might add 
some credibility to the source, whereas a private web page might need to be 
viewed with caution. 
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e The trustworthiness of the site. Does the site convey a sense of serious and 
legitimate purpose? Does it contain suitable disclaimers and explicit state- 
ments about its purpose? 

e How up-to-date the site is. Attention should be given to how regularly the 
pages are updated. Is the date of last updating visible and, crucially, is it 
recent? 

e The popularity of the site. The extent of interest in a website and the size 
of its audience can suggest something about the site’s recognition and 
popularity. These can be gauged by the number of times people have logged 
on to that site - the number of ‘hits’ on the site. 


The credibility of official statistics 


Certain types of official statistics will, to all intents and purposes, provide 
an objective picture of reality. Unfortunately, the same cannot be said for 
certain other types of official statistics. When politicians debate the accuracy 
of unemployment figures or the significance of particular national economic 
figures, there is clearly room for some doubt and controversy about the objec- 
tivity, the accuracy, the completeness and the relevance of some official 
statistics. This should alert us to the point that official statistics cannot always 
be taken as ‘objective facts’. 

To say this is not to reject the use of all official statistics as a source of data for 
social research. But what it does mean is that before treating a set of official 
statistics as accurate and objective, the researcher should consider the following 
factors: 


e The extent to which the event or thing being measured is clear-cut and straight- 
forward. Basically, the more clear-cut the event, the more confidence we can 
have in the data. The official statistics on things like births, deaths, marriages 
and divorce are likely to be virtually complete. Whether or not someone is 
born or dies, whether a marriage or divorce occurs, does not require much in 
the way of interpretation by the registrars involved. The events themselves 
happen or don’t happen, with little scope for ‘creative accounting’ or biased 
interpretation. Alright, a very small number of births and deaths might go 
unrecorded and slip through the net of the tight official procedures for 
collecting such data. But the numbers will be sufficiently low not to present 
any serious challenge to the idea that these official statistics are a full and 
accurate record. Things like unemployment, homelessness and ill health, 
though, are far less clear-cut. There are a number of ways of defining these 
things, and this opens up the possibility of disputes about whether the 
statistics depict real levels of unemployment, etc., or whether they offer a 
picture biased by the nature of the definition that has been used. 


DOCUMENTS 225 


¢ Whether there are vested interests in the statistics that are produced. When those 
who produce the statistics stand to gain or lose on the basis of what the 
figures reveal, the astute researcher should treat them with some caution. 
Trade figures, hospital waiting lists, sales figures: all have consequences 
for people who want or need the statistics to reveal a particular trend. 
And official statistics are not immune to the point. Governments can have 
a vested interest in a variety of statistics, from trade to inflation, from 
unemployment to health. 

e The extent to which the statistics are the outcome of a series of decisions and 
judgements made by people. The more the statistics rely on decisions and 
choices by those who produce them, the more the eventual official statistics 
become open to being challenged. The more the statistics are the end- 
product of a series of choices made by people - choices that involve judge- 
ment and discretion — the more the official statistics can be regarded as 
a ‘social construction’ rather than a detached, impartial picture of the 
real world. 


Crime statistics provide a good illustration of the construction of official stat- 
istics. Crime statistics are based on those crimes that people decide to report 
to the police. A large proportion of crime, however, never gets reported. Petty 
crimes may go unreported. Theft is frequently not reported, particularly when 
people see no chance of recovering their property and when the property 
is not insured. (If a claim is to be made through insurance, a ‘crime number’ 
is needed from the police, and this motivates people to report the crime.) 
Many crimes are not discovered and so obviously do not get reported (e.g. 
embezzlement). There are the ‘crimes with no victims’, where, unless someone 
is caught by the police, no record of the offence will ever appear (e.g. recre- 
ational drug use). In the case of some crimes, there may be an unwillingness 
of the victim to complain because of the accruing stigma (e.g. rape) or fear of 
retribution. 

Even when a crime is reported to the police, there is a level of discretion 
open to them about how they interpret things and whether to formalize 
the matter. A lot of ‘crime’ is sorted out informally, without ever entering the 
system as a recorded crime. Even when it is in the system, the nature of the 
offence is open to some degree of interpretation. From the police officer receiv- 
ing the report onwards, there is some scope for leeway and interpretation 
about (1) whether to prosecute; (2) what kind of offence gets noted; (3) 
whether a caution is given; (4) whether the matter proceeds to court; and (5) 
whether the prosecution is successful or not. At each stage, people are taking 
decisions, using discretion and making judgements. Overall, these can have 
a marked impact on the end-product: the official statistics. It means that 
there will be a huge difference (in the case of crime) between the real level of 
crime in society and the figures that get published by the government as ‘the 
crime rate’. It is estimated that recorded crime may be as little as one-fifth of 
‘actual’ crime. 
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The credibility of records of meetings 


Records of meetings, whether publicly available or on restricted access, pur- 
port to depict things that have happened in a full and accurate manner. 
However, there is ample evidence that such records tend to be partial - in 
both senses of the word. They will tend to be selective in terms of what they 
report, emphasizing some things and ignoring others, and thus recording 
only part of the overall event. They will also tend to reflect a particular 
interpretation of what happened, recording events from a particular angle. 
We should remember that when such records are produced as part of public 
accountability, there will be a tendency to be cautious about what is recorded. 
Things might have been said ‘off the record’ or following ‘Don’t minute this 
but ...’. The records, in other words, may be subtly edited to exclude things 
which might render people vulnerable to criticism when the record is pub- 
lished. The researcher, therefore, needs to be cautious about accepting such records 
at face value. Publicly available records reflect upon matters in a way that is 
publicly acceptable at a given time and in a given social sphere. They tend to 
offer a version of reality massaged to meet public expectations. 


Good practice: always assess the credibility of documentary sources 


Before being used in research, documents should always be checked in terms 
of the credibility of their origins, their purpose and their author. 


Image-based research 


The documents referred to so far have been text-based. There are, however, 
alternative kinds of documentary data available to social researchers, ones that 
are based on visual images. Central to the idea of image-based research, visual 
images can be used as data in their own right —- distinct from text, numbers or 
sounds as a potential source of research information. Just like other documents, 
visual images can prove to be valuable for the purposes of the research in 
terms of: 


e the factual information they contain; 
e how they represent things (the symbolism and hidden meanings com- 
municated through the document or image). 
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Types of image 


For practical reasons there has been a tendency to concentrate on two- 
dimensional ‘still’ images, such as photographs, as the source of image-based 
data. Photographs are relatively inexpensive. They also lend themselves to 
analysis and reproduction alongside more conventional text-based research, 
e.g. in printed journals or academic dissertations. Potentially, though, there 
are a wide variety of visual images that could be used. There have been adven- 
turous attempts on occasion to explore the possibilities of moving images 
and even three-dimensional objects (Emmison and Smith 2000; Banks 2001) 
(Table 12.2). 


Table 12.2 Potential sources of image-based documentary data 


Still Movie Object 
Photographs, Video recording, Cultural artefacts, 
advertisements, archive film clothing and fashion 
drawings, graffiti items, built 


environment and 
places, body signs/ 
language 


The use of ‘created’ images 


The researcher can generate images specifically for the purposes of the 
investigation — so-called created images. The visual images, in this sense, pro- 
vide primary source data. These images can be valuable as a means of recording 
things. Researchers can make records of events, people, cultures, and so on by 
photographing, filming or drawing them. Such visual records provide an 
alternative to tape recordings (sound), an alternative to the use of written 
documents such as field notes, diaries or minutes (text) and an alternative to 
the use of quantitative data such as figures from questionnaires or statistics 
based on systematic observation (numbers). 


Link up with The analysis of image-based data, pp. 294-6 A 
"ee 
When embarking on the use of created images the researcher ought to bear 
in mind certain practical issues. The first concerns equipment. Cameras for 
photographs and camcorders for videos come in a range of technical specifica- 


tions and prices. They can produce output in film or digital format. The 
researcher needs to decide what equipment will best suit the purposes of the 
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research and what equipment is actually available. This may well call for 
some sort of compromise decision that balances the quality and format of 
the output against the resources that exist. In arriving at the decision, the 
researcher will need to be aware that more sophisticated equipment might call 
for technical skills and expertise and that any additional technical wizardry 
attached to more expensive equipment might be wasted without the necessary 
talent to make use of it. Weighed against this, cheap equipment that produces 
poor quality images might prove to be a poor saving of resources. Forethought 
needs to be given to the environments in which the images will be collected. 
Will the lighting be adequate to avoid flash photography that might be 
intrusive or distracting? Will better quality film and equipment overcome such 
a problem? Will a zoom lens be necessary to capture enough detail? 

Quite apart from the technical considerations involved with image-based 
research, when researchers set out to capture data by ‘creating’ the images first 
hand they are confronted as well by a range of social and cultural factors 
that can have a marked impact on their ability to record things ‘as they are’. 
Even when things might seem quite straightforward, there can be problems 
lurking just beneath the surface. For example, filming objects such as buildings, 
transport facilities and artefacts would appear to be fairly unproblematic as a 
means of data collection. There are circumstances, however, where filming and 
photography are not allowed. Some countries might regard it as a security 
threat depending on the location. There are issues of privacy to be considered, 
and in galleries and museums there are often restrictions on photography and 
filming of the artefacts. Forethought needs to be given to the legal and cultural 
context within which data collection is to take place in order to minimize the 
risk that the filming or photography of specific objects will be regarded as an 
offence or as offensive. 

When the image-based research involves people, there is all the more need 
for sensitivity to the social and cultural context. As Prosser and Schwartz 
(1998: 119) point out, researchers need to appreciate that ‘making pictures can 
be a threatening act (amply demonstrated by the metaphors photography 
invokes: we “load”, “aim” and “shoot”).’ The act of being ‘captured’ on 
film can be perceived as threatening even if the researcher’s intention is benign, 
and this likelihood has two important repercussions for the process of data 
collection: 


e It can limit access to research sites. If people feel threatened, if they feel 
that there is an invasion of privacy, they might well refuse to co-operate 
with the research. 

e It can cause people to become self-conscious and to act in a way that they 
would not do normally. 


There is, in effect, an element of reflexivity involved with the collection of 
‘created’ images. Prosser (1998: 104-5) writes of this in terms of ‘procedural 
reactivity’ and ‘personal reactivity’. Procedural reactivity refers to the way the 
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act of taking the photographs makes the researcher more obviously visible 
when collecting data for the record, and the way that this can alter the natural 
state of affairs by inhibiting, embarrassing or in some other way altering 
the behaviour and activities of the person(s) being photographed. Personal 
reactivity refers to the way the photographer/researcher can have an effect on 
the situation simply on the basis of their own identity and personal character- 
istics, and their own judgements in terms of the nature of the photograph that 
is taken. An example might be a photographer’s presence when recording 
images at the birth of a child and the way the sex and age of the photographer 
might have a bearing on the types of photograph that are taken and the 
reaction of the mother, the father and nursing staff to being photographed 
during this intimate moment. 

Bearing the issue of reflexivity in mind, researchers need to be conscious of 
the way their actions will be perceived. Consideration needs to be given to the 
impact of things like: 


e The equipment. Flash photography will be more evident than daylight pho- 
tography, and a discreet small digital camera will be less visible than bulky, 
professional camcorder equipment. 

e The situation. Certain events involve film records more normally than others. 
Ceremonial events, for instance, might allow the use of created images 
by the researcher in a way that would not be the case in more personal, 
intimate circumstances. 

e The people involved. People who live ‘in the public eye’, those with high- 
profile jobs and people like media celebrities, will probably be fairly com- 
fortable in the knowledge that they are being filmed or photographed 
because it is a routine part of their working lives. They might, however, 
reserve the right to privacy on occasions when they are ‘off duty’, regarding 
this as their personal and private life. 

e Personalities. Extroverts and publicity seekers will be likely to adore the 
attention of the camera in a way that more introverted and camera-shy 
people will not. 


The factors come into play, of course, only when people are aware that they 
are being filmed. This need not always be the case. The researcher might 
choose to undertake covert research, in which the process of filming is kept 
secret from those who are being studied. There are particular ethical issues 
that arise in this case and, to gain approval from any relevant authorizing 
body (e.g. an ethics committee), a strong case will need to be made for not 
seeking informed consent from those being studied. This pertains, of course, to 
a range of social research methods, but the fullness of the data captured on 
film can make people all the more defensive and reticent to co-operate - more 
than would be the case if the research relied on things like interviews or 
questionnaire data. 


230 THE GOOD RESEARCH GUIDE 


Link up with Research ethics, Appendix 1 A 


The use of ‘found’ images 


The social researcher can make use of images that already exist — so-called 
found images. These are images that have been produced by other people for 
reasons not directly connected with the researcher’s investigation. 

The kind of images used depends largely on the subject discipline of the 
researcher. Media studies and marketing often use advertisements and news- 
paper photographs. Anthropologists, historians and sociologists will use 
images of groups and cultural events portrayed using film, photographs, paint- 
ings, graffiti or artefacts. Such items contain a visual record that can be used to 
provide factual information about groups or events, or the images can be 
interpreted to provide some better understanding of the ideas and lifestyles 
that generated the images. 

As a source of data, ‘found’ images sidestep a number of problems associated 
with the data gathering process for ‘created’ images. Technical issues about 
equipment and practical issues of access to locations, for instance, no longer 
arise, since the image already exists. Immediately, this suggests that the use of 
found images is an attractive proposition for the social researcher because it 
requires less skill to produce and may well prove less expensive if it saves 
fieldwork, travel and production costs. Equally attractively, it overcomes the 
ethical concerns of getting informed consent in those cases where the 
image involves living people. Added to this, the Internet has vastly increased 
the availability of images for social researchers. Images can be downloaded 
from across the globe to arrive almost instantaneously with the researcher. 
No fieldwork and no travel are required. 

The ease with which found images can be obtained should not seduce the 
researcher into opting for their use without recognizing that there are also 
some specific issues that arise in relation to the use of found images that need 
to be weighed against the ease and cost factors. 


Authenticity 


As with other documentary data, the social researcher who uses visual images 
will want to feel certain about the authenticity of the image. This is particularly 
the case with ‘found’ images that are to be used as a source of factual informa- 
tion. The researcher needs to be wary about whether the image has been 
tampered with, changed or edited from the original. Computer software, 
of course, makes this increasingly simple to do. Photographic images, in 
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particular, are relatively easy to alter and the researcher should not take it as a 
matter of trust that the image is genuine. As Loizos (2000: 95) warns, 


One fallacy is implied by the phrase ‘the camera cannot lie’. Humans, the 
agents who wield cameras, can and do lie: . . . they can distort the eviden- 
tial recording capacity of visual data just as readily as they can distort 
written words. 


Copyright 


Ownership of an image is an important issue, particularly if the image is to be 
reproduced and made publicly available through the process of research. 
Where the image has been created by the researcher, there is generally not too 
much of a problem. The image is usually owned by the researcher. However, 
this may not always be the case. In some cases ownership of the image might 
be claimed on behalf of the person or event portrayed. However, as Banks 
(2001: 168) notes, this issue is ‘relevant largely to representations of celebri- 
ties, those deemed likely to have a financial interest in the use of their 
representations’. 

If the image already exists, the chances are that it ‘belongs’ to some per- 
son, company or institution, in which case it is necessary to obtain permis- 
sion to use the image. At one extreme this might be a fairly informal thing. 
If, for example, the image to be used is a personal photograph from a family 
album, the necessary permission could involve little more than getting 
approval (preferably in writing) from the owner for the use of the image for 
research purposes. At the other extreme, images based on advertisements 
or works of art, and drawn from news media, entertainment magazines or 
the Internet, are the kind of thing that will probably require a very formal 
request to the copyright holder for permission to reproduce the image. If 
the copyright holder agrees to this, it might be conditional on the pay- 
ment of a fee. The fee can be prohibitively large and it is important to 
make it clear when requesting permission that the image is to be used for 
the purposes of small-scale, private research from which there will be no 
monetary gain. Fees might be waived or reduced under these circumstances, 
though this will depend on the nature of the image and the policies of the 
particular copyright holder. If the image to be used has an obscure origin 
and the copyright holder cannot be traced, there is a standard solution to 
the problem, which consists of making a clear statement at the start of any 
report or dissertation along the lines that ‘Efforts have been made to obtain 
permission from copyright holders of the (photographic images) used. Where 
the copyright owners have not been traced the author invites further infor- 
mation from anyone concerned and will seek to remedy the situation.’ This 
does presume, of course, that rigorous efforts have indeed been made and 
the statement should not be used as a way to avoid the work of tracing 
copyright holders. 
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Good practice: copyright and visual images 

Special care should be taken to ensure that the use of visual images for 
research purposes, especially those downloaded from the Internet, do not 
infringe copyright laws. 


/\ Caution: using image-based data 


Despite claims that contemporary society has more of a ‘visual culture’ 
than in the past, and despite some signs of a growing interest in ‘visual 
research’ (Flick 2002), the use of image-based data remains relatively 
uncommon in the social sciences. Part of the reason for this is that 
film, video and three-dimensional images do not readily fit the medium 
within which research data and analyses are normally presented. They 
may not fit in dissertations, journals or computer documents that 
favour a two-dimensional, text-based format. Partly, this might be 
because of the cost of using full-colour glossy photographic data, or 
the space that photographic data might consume when there are tight 
editorial restrictions on the number of pages available. What is clear, 
however, is that image-based research remains somewhat marginal- 
ized and, as yet, struggles to achieve a parity of status in the social 
sciences with research whose data take the form of numbers or words. 
The project researcher undertaking small-scale research might do well 
to bear this in mind. 


Advantages of documentary research 


e Access to data. Vast amounts of information are held in documents. Depend- 
ing on the nature of the documents, most researchers will find access to the 
sources relatively easy and inexpensive. 

e Cost-effective. Documentary research provides a cost-effective method of 
getting data, particularly large-scale data such as those provided by official 
Statistics. 

e Permanence of data. Documents generally provide a source of data which is 
permanent and available in a form that can be checked by others. The data 
are open to public scrutiny. 


DOCUMENTS 233 


Disadvantages of documentary research 


¢ Credibility of the source. The researcher needs to be discerning about the 
information they use. Researchers need to evaluate the authority of the 
source and the procedures used to produce the original data in order to 
gauge the credibility of the documents. This is not always easy. Internet 
documents, in particular, need special scrutiny. Information found on the 
Internet can be up-to-date and good quality stuff. However, just as easily, 
it can be out-of-date, poor quality material because there is little control 
over what is placed on the Internet. From the academic researcher’s pers- 
pective, the worry is that ‘the Internet is characterized by uncontrolled 
and unmonitored publishing with little peer review’ (Cline and Haynes 
2001: 679). 

e Secondary data. When researchers use documents as a source of data, they 
generally rely on something which has been produced for other purposes 
and not for the specific aims of the investigation. 

e Social constructions. Documents can owe more to the interpretations of those 
who produce them than to an objective picture of reality. 
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Checklist for the use of documents (text and 
visual images) 


When undertaking documentary research you should feel confident about 
answering ‘yes’ to the following questions: 


Am | satisfied that the documents/images are genuine in 
terms of what they purport to be (not drafts, forgeries, misleading 
second versions, etc.)? 


Have | considered the credibility of the documents or 
images in terms of: 


their source (published book, journal article, 
official statistics)? 

their author or creator (status, role, ina 
position to know)? 

their sponsorship (organization, funding, 
pressure group)? 

the accessibility of the information they use 
(public domain, restricted, secret)? 


Have website sources been evaluated in terms of their 
accuracy and how recently they have been updated (where 
applicable)? 


Am | satisfied that | have taken account of possible bias 
in the document or image arising from: 


its purpose (description, theory, persuasion)? 

how representative it is (typical, extreme)? 

the editing and selection of extracts used in it? 

its interpretation of facts or theories? 

sensitivity to the information or image it contains? 


Have | provided full details of the sources of the texts or 
images used? 


Has the research avoided contravening copyright laws? 


© M. Denscombe, The Good Research Guide. Open University Press. 


ANALYSIS 


The aims of analysis 


The purpose of analysing something is to gain a better understanding of 
it. Through a detailed examination of the thing that is being studied the aim 
is either: 


e to describe its constituent elements; 
e to explain how it works; or 
e to interpret what it means. 


A description can provide the basis for research in its own right. There are 
types of social research such as ethnography and phenomenology where 
the researcher’s aim is to portray particular cultures or experiences in depth 
and in detail, allowing the texture and subtlety of the situation to become 
visible to the reader. In this case, the description is valuable in its own right 
through the way it allows new insights to be gained. On other occasions, 
descriptions are used as a starting point for research — a platform for a sub- 
sequent analysis based on explanation or interpretation. The logic here is that 
there needs to be a clear vision of what the thing entails before it is possible to 
give an accurate explanation or interpretation of how it works. Either way, 
whether the description is to be used in its own right or as a prelude to further 
investigation, the emphasis is on providing details about: 


e What the thing looks like: measurements, components, appearance, dur- 
ation, etc. 

e When it happened: dates, context of other events, etc. 

e Who was involved: the individuals, the demographics of the sample/popu- 
lation, etc. 

e How often it happens: frequency, duration, etc. 


An explanation looks for rules and regularities that underlie the occurrence 
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of particular social phenomena. The aim is to find out how things work 
by looking for evidence of cause-effect relationships in the data. Once the 
researcher has discovered the causes of specific behaviour, events or inter- 
actions, there is the possibility of predicting how and when things might 
happen in the future — a prospect which has proved very enticing for discip- 
lines such as economics and psychology but which has also been relished 
by many researchers in areas of health, education and business studies. What 
these researchers focus upon are things like: 


e How things are connected: associations and correlations, etc. 

e Why things happen: underlying mechanisms, causes, etc. 

e When things will happen: probability of recurrence, projections for the 
future, etc. 


Interpretation, like explanation, is interested in patterns and regularities that 
lie behind the occurrence of social phenomena. The difference is that interpre- 
tive researchers see their analysis as a matter of providing an understanding 
rather than providing something that is an objective, universal truth. Interpre- 
tivists are still interested in gaining knowledge about how and why things 
happen, but they are sceptical about the prospects of being able to provide 
explanations that can ever be purely objective. They argue that, to a greater or 
lesser extent, theories must inevitably reflect the cultural and historical context 
within which the research takes place and be shaped by the personal values and 
experiences of the researcher. Any theories that are produced, it follows, will be 
‘value-laden’ rather than objective. The consequence of such reasoning is that 
for interpretivists the aim of analysis is to produce a viable account of how 
things work (rather than discover regularities that exist objectively in the social 
world). Bearing this in mind, they tend to focus on things like: 


e How and why things happen: in terms of possible mechanisms, potential 
causes, plausible theories, etc. 

e Who undertook the research: researcher biography, identity and 
experiences; 

e When and where the study took place: cultural and historical context, social 
values; 

e What alternatives exist: comparisons with competing theories. 


Good practice: combining types of analysis 


Small-scale research is likely to use description on its own, or combine 
description with either explanation or interpretation. The project researcher 
should be cautious about trying to combine explanation and interpretation 
because these two types of analysis tend to operate on different visions of 
social reality and the possibilities for understanding it. 
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Approaches to analysis: quantitative and 
qualitative research 


There are many kinds of analysis the social researcher can use in order to 
describe, explain or interpret the data. In practice, though, the options tend 
to gravitate around the notions of ‘quantitative’ and ‘qualitative’ research. 
These terms are widely used and commonly understood within social research, 
and they have come to represent contrasting alternatives that signify distinct, 
some would even say incompatible, stances on a broad range of beliefs and 
practices relating to social research. Initially, the contrast concerns the differ- 
ent kinds of data that they use and the implications of this are explored in 
Chapters 13 and 14. In a nutshell, the distinction between the approaches 
centres on the fact that: 


e quantitative research uses numbers as the unit of analysis; 
e qualitative research uses words or visual images as the unit of analysis. 


Linked with this, the terms have also come to be associated with a particular 
set of assumptions about the nature of the social world. Quantitative research 
tends to be linked with positivism and qualitative research tends to be associ- 
ated with interpretivism. This division is far from watertight but it continues to 
work as an initial assumption about the respective stances of quantitative and 
qualitative research. Quantitative research is associated with the production of 
numerical data that are ‘objective’ in the sense that they exist independently 
of the researcher and are not the result of undue influence on the part of the 
researcher himself or herself. Ideally, the numerical data are seen as the prod- 
uct of research instruments that have been tested for validity and reliability to 
ensure that the data accurately reflect the event itself, not the researcher’s 
preferences. Qualitative research, by contrast, tends to place emphasis on the 
role of the researcher in the construction of the data. There is typically little 
use of standardized research instruments in qualitative research. Rather, it 
is recognized that the researcher is the crucial ‘measurement device’, and 
that the researcher’s background, values, identity and beliefs might have a 
significant bearing on the nature of the data collected and the analysis of that 
data. In essence, this means that: 


e quantitative research tends to be associated with researcher detachment; 
e qualitative research tends to be associated with researcher involvement. 


The terms are also associated with visions about the size, focus and design 
of investigation that is required. Quantitative research tends to favour larger- 
scale research with larger numbers and greater quantities. The sheer amounts 
involved, once they have been transformed into numerical data, do not slow 
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down the analysis to any discernible degree. A statistical procedure can be 
undertaken by a computer on a sample of 1,000 pretty much as quickly as it 
can on a sample of 100. The results, though, will be more reliable when con- 
ducted on 1,000 than on 100. The larger the numbers involved, the more the 
results are likely to be generalizable and reliable, statistically speaking. By con- 
trast, qualitative research tends to involve relatively few people or events. 
Much of this reflects the preference for depth of study and the associated ‘thick 
description’ which only becomes possible in relation to limited numbers. 
Qualitative researchers like to be ‘close to the data’ and like to have a detailed 
familiarity with the data in order to conduct the analysis. Their preference for 
small-scale studies also reflects the fact that words do not lend themselves to 
the kind of analysis that can utilize the power of computers in the same way 
that numbers do. For the qualitative researcher, there is a vast difference in the 
time it takes to analyse results from larger amounts of data compared with 
smaller amounts. For these reasons: 


e quantitative research tends to be associated with large-scale studies; 
e qualitative research tends to be associated with small-scale studies. 


The terms quantitative and qualitative research also imply a distinction in 
terms of the breadth of focus of an investigation. In the case of quantitative 
research there is a tendency to focus on specific variables and to study them, if 
not in isolation, at least with reference to a limited range of other variables. 
Qualitative research, on the other hand, generally exhibits a preference for 
seeing things ‘in context’ and for stressing the importance of multiple inter- 
relationships between a wide range of factors in operation at any one time 
in the setting. Qualitative research tends to operate on the assumption that 
social ‘realities are wholes that cannot be understood in isolation from their 
contexts, nor can they be fragmented for separate study of their parts’ (Lincoln 
and Guba 1985: 39). As a generalization, then: 


e quantitative research tends to be associated with analysing specific variables; 
e qualitative research tends to be associated with a holistic perspective. 


Finally, the notions of quantitative and qualitative research carry with 
them an image of contrasting approaches to research design. Quantitative 
research tends to be associated with precise research designs in which the 
process of analysis is clearly separated from the process of data collection. 
The research questions or hypotheses are carefully formulated. After this 
comes the process of data collection which usually involves a predetermined 
sample to be collected or experimental procedure to be undertaken. Finally, 
there comes the data analysis stage in which the data that have been previ- 
ously collected are subjected to statistical tests or other quantitative analysis. 
In contrast with this, qualitative research tends to be linked with the idea 
that research questions should not, and cannot, be specified precisely in 
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advance of the data collection stage. Nor can data analysis be hived off to a 
period after the data collection has been completed. As exemplified by the 
grounded theory approach (see Chapter 6), this style of qualitative research 
sees the formulation of the research question, the collection of data and the 
analysis of data as an emerging and iterative process. In effect, the analysis 
commences at the earliest stage of research and continues throughout the 
whole time of the investigation. The distinction, then, revolves around the 
way that: 


e quantitative research tends to be associated with data analysis after data 
collection; 

e qualitative research tends to be associated with data analysis during data 
collection. 


/\ Caution: the distinction between quantitative and 
qualitative research 


In the real world of social research things may not fall neatly into the 
two categories. There are three main reasons for this: 


1 In theory, the distinction is too simplistic. The assumptions associ- 
ated with the two approaches are sometimes shared and sometimes 
overlap. 

2 In practice, many researchers use a mixed methods approach which 
combines quantitative and qualitative research rather than treating 
them as incompatible alternatives. 

3 Strictly speaking, the distinction between ‘quantitative’ and 
‘qualitative’ should relate to the treatment of data, rather than types 
of research as such. 


Stages of data analysis 


The analysis of research data tends to follow a process involving five stages. 
The stages can be seen in relation to both quantitative and qualitative data, as 
Table II.1 shows. However, it also shows that there are differences between 
quantitative and qualitative data analysis, with quantitative approaches tend- 
ing to shape their data more consciously and more explicitly in the earlier 
stages of the process compared with qualitative approaches. More details on 
the specific approaches to data analysis are provided in Chapters 13 and 14. 
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Table Ill.1 The five main stages of data analysis 


Quantitative data 


Qualitative data 


1 Data preparation 


2 Initial exploration of 
the data 


3 Analysis of the data 


4 Presentation and 
display of the data 


5 Validation of the data 


Coding (which normally takes 
place before data collection) 
Categorizing the data 
Checking the data 


Look for obvious trends or 
correlations 


Use of statistical tests (e.g. 
descriptive statistics, factor 
analysis, cluster analysis) 
Link to research questions or 
hypotheses 


Tables 

Figures 

Written interpretation of the 
statistical findings 


External benchmarks. Internal 
consistency. Comparison with 
alternative explanations 


Cataloguing the text or visual 
data 

Preparation of data and loading 
to software (if applicable) 
Transcribing the text 


Look for obvious recurrent 
themes or issues 

Add notes to the data. Write 
memos to capture ideas 


Code the data 

Group the codes into categories 
or themes 

Comparison of categories and 
themes 

Look for concepts (or fewer, 
more abstract categories) that 
encapsulate the categories 


Written interpretation of the 
findings 

Illustration of points by quotes 
and pictures 

Use of visual models, figures 
and tables 


Data and method triangulation 
Member validation 
Comparison with alternative 
explanations 


Source: Adapted from Creswell and Plano Clarke (2007: 129), Table 7.1. 


13 


QUANTITATIVE DATA 


Sources of quantitative data e Types of quantitative data e Preparing 
quantitative data for analysis e Grouping the data e Describing the 
mid-point or average e Describing the spread of data e Looking for 
patterns and relationships in the data e Statistical tests for association 


and difference e Presenting the data — tables and charts e Validating 
the data e Advantages of quantitative analysis e Disadvantages of 
quantitative analysis e Checklist for the use of basic statistics 


This chapter focuses on basic aspects of the analysis and presentation of 
quantitative data. There is no attempt in this book to cover more advanced 
statistical procedures that use quantitative data for testing hypotheses or 
forecasting outcomes. Such ‘inferential’ statistics are valuable, but they require 
a level of familiarity with statistics that project researchers might not have the 
time or inclination to acquire. For those who want to study the statistical 
analysis of quantitative data in more depth, there are many textbooks and 
online resources that are readily available, and reference should be made to 
such sources. The majority of project researchers, however, will prefer to 
restrict their analysis to the use of basic descriptive statistics. The premise of 
this chapter is that, properly used, simple ‘descriptive’ statistics can offer the 
project researcher a straightforward, yet rigorous way of: 


e organizing the data; 

e summarizing the findings; 

e displaying the evidence; 

e describing the profile of findings (how the data are distributed); 

e exploring connections between parts of the data (correlations and 
associations). 
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Good practice: the use of numbers in small-scale research 


‘Every researcher can and should use numbers in their research’ (Gorard 
2006: 2). Quantitative data can be used very effectively without the need for 
complex statistical analysis. 


Sources of quantitative data 


Quantitative data take the form of numbers. They are associated primarily with 
strategies of research such as surveys and experiments, and with research 
methods such as questionnaires and observation. These are not, however, the 
only sources of quantitative data. For example, the use of content analysis with 
texts (such as interview transcripts) can also produce numerical data. The kind 
of research method used, then, is not the crucial thing when it comes to defin- 
ing quantitative data. It is the nature of the data that the method produces 
that is the key issue. It is important to bear this point in mind and to recognize 
that quantitative data can be produced by a variety of research methods, as 
Table 13.1 shows. 


Table 13.1 Sources of quantitative data 


Numbers Research method 
Answers to closed-ended questions Questionnaires 
Content analysis of transcripts Interviews 
Measurements from experiments | Öbséivation 
Observation schedule used with events 


Official statistics (health, education, trade, etc.) 
Business data (performance, employment, etc.) Documents 
Content analysis of, for example, company reports 


Types of quantitative data 


Numbers can represent different types of things, and social researchers who 
want to use quantitative data need to be absolutely clear about the type of 
numerical data they are using. This makes a major difference to what can be 
done with the data, statistically speaking, and the kind of conclusions that can 
be drawn from the analysis. There are certain statistical techniques that work 
with some kinds of data that will not work with others. In this respect, numer- 
ical data can be considered under six headings. The first four concern the 
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distinction established by Stevens (1946) between four types of measurement. 
These are called nominal, ordinal, interval and ratio. In addition to this, 
researchers should also be aware of the difference between ‘discrete’ and 
‘continuous’ data. 


Nominal data 


Nominal data come from counting things and placing them into a category. 
They are the lowest level of quantitative data, in the sense that they allow little 
by way of statistical manipulation compared with the other types. Typically, 
there is a head count of members of a particular category, such as male/female 
or White/South Asian/African Caribbean. These categories are based simply on 
names; there is no underlying order to the names. 


Ordinal data 


Like nominal data, ordinal data are based on counts of things assigned to 
specific categories, but, in this case, the categories stand in some clear, ordered, 
ranked relationship. The categories are ‘in order’. This means that the data in 
each category can be compared with data in the other categories as being 
higher or lower than, more or less than, etc., those in the other categories. The 
most obvious example of ordinal data comes from the use of questionnaires in 
which respondents are asked to respond on a five-point scale such as: 


Strongly Agree Neutral Disagree Strongly 
agree disagree 

l l l l | 

1 2 3 4 5 


The responses coded as 2 (agree) can legitimately be seen as more positive than 
those coded as 3, 4 or 5 (neutral, disagree, strongly disagree), but less positive 
than those coded 1 (strongly agree). This scale is known as the Likert scale. It is 
worth stressing that rank order is all that can be inferred. With ordinal data we do 
not know the cause of the order, or by how much they differ. 


Interval data 


Interval data are like ordinal data, but the categories are ranked on a scale. This 
means that the ‘distance’ between the categories is a known factor and that it 
is proportionate. This means that the researcher not only can deal with the data 
in terms of ‘more than’ or ‘less than’, but also say how much more or how 
much less. 

Calendar years provide a suitable example of such data. Data collected for 
the years 1966, 1976, 1986, 1996 and 2006 not only differ in terms of being 
earlier or later than one another, they are also earlier or later by a known 
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timespan interval. This allows the researcher to use addition and subtraction 
(but not multiplication or division) to contrast the difference between various 
periods: the difference between 1976 and 1986 can be directly compared with 
the difference between 1996 and 2006, and so on. 


Ratio data 


Ratio data are like interval data, except that the categories exist on a scale 
which has a ‘true zero’ or an absolute reference point. When the categories 
concern things like incomes, distances and weights, they give rise to ratio data 
because the scales have a zero point. Calendar years, in the previous example, 
do not exist on such a scale, because the year 0 does not denote the beginning 
of all time and history. The important thing about the scale having a true zero 
is that the researcher can compare and contrast the data for each category in 
terms of ratios, using multiplication and division, rather than being restricted 
to the use of addition and subtraction as is the case with interval data. 
Ratio data are the highest level of data in terms of how amenable they are to 
mathematical manipulation. 


Discrete data 


Certain data are based on phenomena which naturally come in whole units. 
Numbers of children per family, in reality, have to be whole numbers. We might 
aggregate the numbers to arrive at an average figure of 1.9 or whatever, but we 
do not suppose that there exists anywhere 0.9 of a child belonging to a family. 
In this example, our measurement of children per family can be exact for each 
and every household, with no need for approximations or results accurate to 
the nearest fraction. The discrete data come in chunks: 1, 2, 3, 4, and so on. 


Continuous data 


Contrasted with this there are certain kinds of data which, for practical pur- 
poses, are inevitably measured ‘to the nearest unit’ simply because they do not 
come in neat, discrete chunks. Such things are measured to the nearest small 
unit because, as a variable, they are continuous. People’s height, age and weight 
are obvious examples here. In principle, there is no end to the precision with 
which we might try to measure these items. We could measure height in milli- 
metres, weight in grams, age in seconds. For practical purposes, however, such 
precision would be counter-productive, because it would throw up vast arrays 
of data which would not lend themselves to analysis. When variables are con- 
tinuous, the researcher, in effect, needs to categorize the data. The unit of data 
is not exact height, exact weight, exact age; rather, it is ‘height falling within a 
range of 2 cm’, ‘weight to the nearest 0.5 kilograms’ or ‘age at last birthday’. 
The significance of continuous data lies with the implicit categories which 
are formed by the researcher to cope with difficulties of measuring points/ 
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units on a sliding scale. The researcher needs to be absolutely clear about 
the boundaries to such categories and exactly where the mid-point of the 
category resides. The accuracy of statistics based on continuous data depends 
on knowing the boundaries and mid-point to the categories of the data. 


Good practice: different kinds of numerical data 

Researchers wanting to use quantitative data need to understand what kind of 
data they are working with and appreciate the possibilities and limitations 
associated with that particular kind of numerical data. 


Preparing quantitative data for analysis 


Coding the data 


The raw data with which the social researcher works sometimes occur ‘natur- 
ally’ in the form of numbers. This is convenient. However, on many occasions 
the researcher starts off with material in the form of words or pictures and 
needs to transform the material from this format into the only format suitable 
for quantitative analysis: numbers. This involves a process of ‘coding’ the data. 
Coding, in essence, entails the attribution of a number to a piece of data, or 
group of data, with the express aim of allowing such data to be analysed in 
quantitative terms. Such coding is usually done at an early stage in the 
research process and, in fact, the general advice as far as quantitative analysis is 
concerned is to do the coding prior to the collection of the data. 

If this sounds a bit abstract, it might be useful to illustrate the process briefly 
in relation to the use of questionnaires. Suppose that the researcher is interested 
in people’s occupations, and intends to conduct a survey of 1,000 people. The 
questionnaire might contain a question asking the respondent to state his or 
her occupation. This would result in a mass of data, all in the form of words, cov- 
ering a huge range of possibilities. In this format the data are difficult to handle 
and very difficult to analyse. Imagine the researcher’s problem when faced with 
a list of 1,000 separate jobs. To make the task of analysis easier, the researcher 
needs to identify a smaller number of categories of jobs, and then to assign each 
job listed to one of the categories. Each category will have a numerical code, 
and so the process of analysis involves coding each of the words by giving it a 
numerical value which accords with the category of job to which it fits. 

Where the researcher knows in advance that they want to generate quantita- 
tive data, it makes good sense to sort out the categories and their respective 
numbers and build these into the design of the research. Things like a person’s 
occupation can be categorized in advance using existing official classifications. 
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Instead of people being asked to state what their job is, they can be asked 
to identify which category of occupations best fits their own. The result of 
doing it this way is that, if there are, say, ten categories of occupation used, 
the researcher will obtain results in the form of numbers in each of the ten 
categories, rather than the list of 1,000 words. The data, in this format, are 
ready for analysis. 


Se, 
Link up with Questionnaires, Chapter 9 , 


Grouping the data 


The first stage in the analysis of quantitative data is to organize the raw dataina 
way that makes them more easily understood. As Table 13.2 shows, it is rather 
difficult to make sense of raw data when the number of items goes above ten. 


Table 13.2 Raw data example 
Days absent from work through illness during the previous year, 30 employees 


1 12 (0) 9 4 7 
5 3 6 15 10 8 
4 36 3 13 3 3 
3 6 3 (0) (0) 
10 5 8 (0) 10 2 


Source: Human Resources Dept, Healthy Life Insurance Co. 


Initially, it is some help to construct an array of the raw data, i.e. to arrange 
the data in order. As Table 13.3 shows, this immediately helps in terms of 
getting some grasp of what the data hold. 


Table 13.3 Array of raw data example 
Days absent from work through illness during the previous year, 30 employees 


O 2 3 5 8 10 
(0) 3 3 6 8 12 
(0 3 4 6 9 13 
(0 3 4 6 10 15 
1 3 5 7 10 36 


Source: Human Resources Dept, Healthy Life Insurance Co. 


A further stage in organizing the data is to make a tally of the frequencies (see 
Table 13.4). This gives a clearer picture of which frequencies were the most 
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common and is far better in terms of being able to ‘read’ the data. It immedi- 
ately suggests certain things about the data and invites questions about why 
certain frequencies are more common than others. Is this sheer fluke, or is 
there some underlying cause? 


Table 13.4 Tally of frequencies example 
Days absent from work through illness during the previous year, 30 employees 


Days Frequency Days Frequency Days Frequency 


(0) 4 5 2 10 3 
1 1 6 3 12 1 
2 1 7 1 13 1 
3 6 8 2 15 1 
4 2 9 1 36 1 


Source: Human Resources Dept, Healthy Life Insurance Co. 


Where there are a large number of frequencies, the researcher can organ- 
ize the data by grouping the frequencies (see Table 13.5). Such grouped fre- 
quency distributions are a very valuable and commonly used strategy for the 
organization of raw data. 


Table 13.5 Grouped frequency distribution example 
Days absent from work through illness during the previous year, 
30 employees 


Group Frequency 


0-2 days 

3-5 days 1 
6-8 days 

9-11 days 

12-14 days 

15+ days 


MNNADOD 


Source: Human Resources Dept, Healthy Life Insurance Co. 


/\ Caution: the effect of grouping the data 


It is worth bearing in mind that as soon as researchers create groups 
they are imposing on the data in a pretty strong way. The data in the 
format of a grouped frequency distribution may be easier to under- 
stand, but they have moved away from a natural or raw state and begin 
to bear the hallmarks of the creative shaping of data by the researcher. 
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Describing the mid-point or average 


One of the basic things that a researcher needs to do with a quantitative data 
set is to provide a description of the mid-point or the average. There are three 
ways of doing this and the choice of which measure of central tendency is used 
will depend on the kind of numerical data involved. 


The mean (the arithmetic average) 


The mean is what most people have in mind when, in common parlance, they 
think about ‘the average’. It is a measure of central tendency based on imagin- 
ing what would happen if instead of the values in the distribution all varying 
they were all the same — if there was an equal distribution of values. By adding all 
the values together and dividing by the number of cases it is easy to calculate 
this arithmetically. To calculate the mean of the values below: 


e add together the total of all the values for a variable or category (= 116) 
e divide this total by the number of cases (+ 8) 


This gives a mean of 14.5. 
1 4 7 11 12 17 17 47 


For social researchers, the mean has certain strengths and certain limitations 
when it comes to describing the data. 


e The mean can be used with interval and ratio data. In practice, it is often used 
with ordinal data as well, although this is controversial. Calculations of the 
mean, however, are definitely not appropriate when working with nominal 
data. Clearly it makes no sense to think of the average when the researcher 
is dealing with nominal data such as categories of sex, ethnic group or 
occupations; how can you calculate the average of two sexes, of four ethnic 
groups or of ten occupations? 

e The mean is affected by extreme values (‘outliers’). Because the mean includes 
all values, it is prone to being affected by the odd one or two values that 
might be vastly different from the rest. Extreme or unusual values get 
included in the overall mean, and can shift the figure for the mean towards 
any such ‘outliers’. Two things will minimize the impact of such outliers. 
First, a large data set will shroud the influence of the odd one or two outliers. 
An outlier will have less of a distorting impact on a large sample than it 
will on a small sample, because the extent of its deviation from the rest will 
get subdivided more and more as the size of the sample gets bigger. Second, 
and linked to this, calculations of the mean are safest when there are 


QUANTITATIVE DATA 249 


relatively few outliers, and the outliers that do exist have a tendency to 
balance up and cancel each other out. 

e Use of the mean may lead to strange descriptions. When we conclude, for 
example, that the average size of households in Britain has fallen from 2.91 
persons in 1979 to 2.3 persons in 2000, we clearly do not mean this as a 
literal description of the situation. What does 0.3 of a person look like? It is a 
statistical description. 


The median (the middle point) 


The idea of ‘average’ can sometimes carry with it the idea of ‘middle of the 
range’, and this is the sense in which the median operates. The median is the 
mid-point of a range. 

Calculation of the median is really very straightforward. Values in the data 
are placed in either ascending or descending rank order and the point 
which lies in the middle of the range is the median. With even numbers of 
values, it is the mid-point of the two central values. So, in the following set 
of data, the median is 11.5, half-way between the two middle values. 


1 4 7 11 12 17 17 47 


There are some crucial advantages and limitations connected with the use of 
the median as a measure of central tendency. 


e It can be used with ordinal data as well as interval and ratio data. This 
opens up the scope of uses to many areas which are not suitable for the 
arithmetic mean. 

e Because it is an ordinal operation, the median is not affected by extreme 
values, the ‘outliers’. Looking at the values above, it should be clear that 
the value of 47, an obvious outlier, does not affect the calculation of the 
median. It does, though, affect the calculation of the mean. As far as the 
median is concerned, that value could lie anywhere between 18 and infin- 
ity because its sole significance is that it exists as a value in the range which 
is greater than 17. Now this, of course, can cut both ways. While the 
median is unaffected by outliers, it is equally insensitive to the implica- 
tions of any such extreme values. It effectively eliminates such data from 
the picture. 

e The median works well with a low number of values. In fact, in contrast to the 
mean, it is better suited to low numbers of values. 

e We have the advantage of knowing that exactly half the values are above 
the median and half the values are below the median. It offers a quick and 
easy-to-grasp measure of the central point of the data. 

e The main disadvantage of the median is that it does not allow for further 
mathematical processing. 
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The mode (the most common) 


As the name suggests, when social researchers use the mode as a measure of 
central tendency they have in mind the most fashionable or popular figure. 
But this does not imply that they are looking for the best liked value in the 
data. It is the value which is most common that is the mode. Identification of 
the modal value simply consists of seeing which value among a set occurs most 
frequently: this is the mode. In the set below, the mode is 17. 


1 1 4 4 7 11 #12 17 17 17 47 


As with the mean and the median, there are certain strengths and weaknesses 
pertaining to the use of the mode, and these need to be considered: 


e The mode can be used with nominal data, as well as ordinal, interval and ratio 
data. It has the widest possible scope for application, then, in relation to 
social data. As a measure of central tendency it makes sense, for example, 
to talk about the most commonly watched soap opera on TV, whereas we 
cannot sensibly talk about the mean or median soap opera. 

e Like the median, it is unaffected by outliers or extreme values. It focuses only 
on the most common value and ignores all others. 

e It shares with the median the limitation that the mode does not allow any 
further mathematical processing to be undertaken on it. It is a standalone 
end-product. 

e One other limitation: there might not be one most popular value, or some 
values might share top spot, so that there might be two or more modes. 


Good practice: the difference between means, medians and modes 

When describing data, social researchers need to be aware of the difference 
between the three measures of central tendency and decide which is most 
appropriate for their specific purpose. 


Describing the spread of data 


If researchers wish to describe a set of frequencies it will help if they can pro- 
vide not only a measure of central tendency but also some idea of the way the 
data are spread out from that central point — a measure of dispersion. 
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The range 


This is the most simple way of describing the spread of data. It is none the less 
effective, and a valuable method of giving an instant image of the dispersion 
of the data. The range of the data is calculated by subtracting the minimum 
value from the maximum value. In the set of values below, the range is 44 
(i.e. 47 minus 3). 


3 4 7 11 12 17 #17 47 


Although a statement of the range has value as an instant picture of how 
wide the data are spread, it does have a rather serious limitation as a statistic. 
It is governed by the extreme values in any set and may consequently give 
a biased picture of the spread of the values between the extremes. In the 
example above, the range of 44 is obviously determined by the outlier value 
of 47. Visually, it is evident that seven of the eight values lie within a range of 
just 14. So, although reference to the range of values is quick and easy, it needs 
to be used cautiously and with attention paid to any extreme values and the 
way these might give a misleading impression of the spread. 


Fractiles 


Fractiles are based on the idea of dividing the range of the data into fractions. 
Each fraction contains the same number of data cases. When quartiles are used, 
the data are divided into four sections. Deciles operate on the same principle 
but, as the name suggests, they divide the range into tenths. Where percentiles 
are used, the data are divided into 100 equal parts. 

There are two uses of fractiles for social researchers. First, they can be 
used for the selection of cases to analyse. Researchers can use them as a way 
of eliminating extreme highs and lows from the data they wish to examine. 
For example, by focusing on those cases that come in the second and third 
quartiles, researchers know that they are dealing with exactly half of all the 
cases and, more than that, they are dealing with the half that are ‘in the 
middle’. This middle part of the range is known as the ‘inter-quartile range’. 
With incomes, for instance, use of the inter-quartile range overcomes the 
possibility of some very high incomes distorting the overall picture, since 
these would occur in the top quartile. Likewise, by not using the bottom 
quartile the researchers would eliminate very low incomes from their analysis. 

A second use of fractiles is for comparison. They allow comparisons to be 
made between those cases that occur at one end of the range and those that 
occur elsewhere in the range. The top quarter can be compared with the 
bottom quarter, or the highest 10 per cent with the lowest 10 per cent. The 
most prominent use of this is in comparisons of income and wealth. The top 
decile (the 10 per cent of all earners who earn more than the other 90 per cent 
of earners) can have their pay compared with the 10 per cent of all earners who 
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earn the least. The comparison can be in terms of how much is earned. Here 
the researchers can calculate the mean earnings of the top decile and com- 
pare this with the mean earnings of the bottom decile. Or the proportion of 
all income going to those in the top decile can be compared with the pro- 
portion going to those in the lowest decile. Used in such a way, fractiles are 
a valuable tool for the social scientist in describing the spread in a frequency 
distribution. 


Standard deviation 


An alternative method of dealing with the spread of data comes in the form 
of the standard deviation. This works on the idea that if we wish to understand 
the dispersion of the data, it would be useful to know how far, on average, the 
values varied from the mean. It is not an absolute measure of the difference 
between the highest value and the lowest value, as is the case with the use 
of ranges. More subtly, the standard deviation uses all the values (not just top 
and bottom) to calculate how far in general the values tend to be spread out 
around the mean. 

There are variations in the formulae used for calculating the standard devi- 
ation, and there are certain complications that set in when it is used with 
grouped frequencies. However, these should not detain us because computer 
software will generally take care of the mathematics involved. The basic point 
is this: the greater the dispersion, the larger the standard deviation. 


Example: calculating the standard deviation — the logic 


Technically, we want to know ‘the mean variation from the mean’. In com- 
mon language, we want to know ‘the average amount by which our data is 
spread out from the middle’. To do this, we could add up the amounts by 
which each value on our data list varies from the mean and, to get the 
‘average’, divide this total by the number of values on the list. If we did so, 
however, the result would always equal zero. This is because the positive and 
the negative differences from the mean, when totalled, will always cancel 
each other out. To get around this the differences from the mean need to 
be squared because this ensures that they all have a positive value and 
cannot cancel each other out. When these ‘squares of the differences from 
the mean’ are added together and divided by the number of cases in the 
data list they provide a measure of the ‘average’ spread around the middle. 
Or, to put this in statistical jargon, we can calculate the mean of ‘the sum 
of the squares’ and get a measure of the variance of the data. To get the 
units back to the same as those in the actual distribution being looked at we 
need to get the ‘square-root of the variance’ — and it is this which is known as 
the ‘standard deviation’. 
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Use of the standard deviation as a measure of dispersion has some distinct 
advantages in relation to the analysis of quantitative data. Specifically: 


e The standard deviation lends itself to further mathematical analysis in a 
way that a straightforward description of the spread does not. 

e Following from this, the standard deviation assumes huge importance in 
social research, because it is the basis of so many other statistical procedures. 


However, it is important to remember that it is not a measure of dispersion 
that can be used with nominal data. Because it takes the arithmetic mean as 
the basis of its calculations, only those kinds of data that lend themselves 
to this measure of central tendency can be used. That is, the standard devia- 
tion can be used only with cardinal numbers (interval and ratio data). It is 
meaningless to use it with nominal or even, strictly speaking, with ordinal data. 


Good practice: providing a profile of the data 


A good description of the profile of data normally requires measures of central 
tendency to be used alongside measures of dispersion. 


Looking for patterns and relationships in the data 


Having produced a description of the profile of the data, the next stage of 
analysis is to look for patterns and relationships in the data. This involves 
looking for connections. Researchers will want to check whether there are any 
patterns of association between variables within a data set or whether there are 
any relationships between two or more sets of data. Does one category of data 
match another, or does one category vary along with another? Is there a differ- 
ence between two sets of data where it might have been expected that there 
would be a similarity? 

Looking for connections, at one level, can be a very straightforward process. 
In essence, it need only involve looking at the data in the tables or the charts to 
see what links are evident. However, appearances can be deceptive. They may 
not tell the whole truth. They may not tell the truth at all! Looking at the 
data for connections may be a good starting point but, from the point of view 
of good research, it remains just that — a starting point. Suppose there is an 
apparent connection between two variables. Should this lead the researcher 
to conclude that the association is real and prompt him or her to conduct 
further research on the basis that a link had been discovered between the two 
variables? The answer has to be ‘No, at least not yet.’ 
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For research purposes, the mere appearance of a connection is not enough. 
Good researchers want to know: 


e whether the findings were a fluke; 

e how strong the connection is between the two variables; 

e whether one causes the other, or whether the two are mutually 
interdependent. 


These seem reasonable things to ask of the data. Although we may be able to 
‘see’ a link ina table or chart, we really need something more positive to support 
what appears on the surface to be a link between the variables. We need some 
reassurance that the findings were not a one-off fluke. We need something to 
give us confidence that what we have found is a genuine association of the two. 
And this is where statistical tests of significance prove to be of enormous benefit. 
Such statistical tests provide an estimate of the probability that any association 
we find between two or more variables is the product of sheer chance rather 
than a genuine connection between such variables — connections that will be 
found on other occasions. They provide a benchmark for researchers, indicat- 
ing whether to proceed on the assumption that the apparent connection is real 
or whether to treat the data as unreliable evidence on the point. 

Good researchers are very cautious in their approach to the search for 
patterns and relationships in the data. They approach things with a sceptical 
mindset and start from the premise that any pattern they might detect in the 
data could actually be the outcome of nothing more than sheer chance. As an 
expression of this cautious approach researchers using statistics start with a 
‘null hypothesis’. They approach their findings with the presumption that there 
is no real pattern or no real relationship between sets of data, and they will 
only be persuaded to reject this position if there is ‘overwhelming’ statistical 
evidence to the contrary. In the social sciences there is a convention that 
researchers will not accept that observed patterns or relationships are worthy 
of being considered ‘real’ unless the probability of them arising by chance is 
less than 1 in 20. In technical terms this involves a significance level of p < 0.05. 
Only if the patterns or relationships are shown to be statistically significant 
(p < 0.05) will the good researcher be persuaded to reject the null hypothesis. 
Like members of a jury, the researcher needs to be persuaded beyond reason- 
able doubt and this happens where the evidence/findings is likely to be true at 
least 19 times out of 20. 


Good practice: a cautious approach and the null hypothesis 


Social researchers should start from the premise that there is no real pattern or 
no real relationship between sets of data, and only be persuaded to reject this 
position if the statistics point to a probability of less than 1 in 20 (p < 0.05) 
that the results arose by chance. 
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/\ Caution: the meaning of ‘significance’ 


Statistical significance does not necessarily imply social significance. 
The researcher needs to be wary of mixing the two senses of ‘signifi- 
cance’. Statistical significance only relates to the amount of con- 
fidence we have that the findings we obtained were not the product 
of pure chance. It does not automatically imply that the findings 
are ‘important’. That is something to be shown elsewhere by other 
means. 


Statistical tests for association and difference 


How do | find out if two variables are associated to a significant level? 


Probably the most flexible and certainly the most commonly used statistical 
test for this is the chi-square test. The chi-square test works with nominal data 
as well as ordinal, interval and ratio data - which does much to explain its 
popularity. The columns and rows can use nominal data based on things 
like sex, ethnicity and occupation, factors which feature prominently in data 
collected by social researchers (see Table 13.6). 


Table 13.6 Chi-square and contingency table examples 
Days absent from work through illness during the previous year, by sex of employee 


Sex 
Male Female Total 
Days lost 0-5 days Count 8 7 15 
Expected count 7.0 8.0 15.0 
6 or more days Count 6 g 15 
Expected count 7.0 8.0 15.0 
Total Count 14 16 30 
Expected count 14.0 16.0 30.0 
Value d.f. (2-sided) 
Pearson chi-square 0.536* I 0.464 


Note: * O cells (0.0%) have expected count less than 5. The minimum expected count is 7.00. 
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It works on the supposition that if there were no relationship between the 
variables then the units would be distributed in the cells of the contingency 
table in equal proportion. This means that if there were three times as many 
women as men in a sample, we would start from the assumption that women 
would account for 75 per cent of the amounts or frequencies we are measuring — 
for instance, absences from work. If there is no connection between sex and 
absenteeism from work, we might expect the larger proportion of women in the 
sample to account for a larger proportion of absences — in the strict ratio of 3:1. 
This ‘expected’ distribution (based on the null hypothesis) is, in other words, a 
theoretical distribution based on a starting premise that the two variables con- 
cerned do not affect each other. In reality, it is extremely unlikely that the 
research findings will match this theoretical distribution. Almost certainly there 
will be a difference between what might be expected and what is actually found 
in practice. And this difference, the difference between what was ‘observed’ and 
what was ‘expected’, is the key to the chi-square test. The chi-square test uses 
the extent of difference (in the cells of a contingency table) between what was 
observed and what might have been expected in order to calculate whether 
we can have confidence that the observed relationship was actually due to 
something other than pure chance - whether it was real or a fluke. 

The main restriction on the use of the chi-square test is that there needs to be 
a sufficiently large data set and/or a sufficiently even distribution among the 
various categories in order to get a minimum of five in each cell. Where cells in 
the contingency table have fewer than five in them, it threatens the accuracy 
of the statistic. There are ways around this should it become a problem. There 
is a statistical fix which can be used (e.g. Yates correction factor), and most 
computer programs will put this into operation automatically. Alternatively, 
the categories can be combined and reduced in number to ensure that each cell 
has enough data in it. 

There is a danger with the use of the chi-square test that the results are heavily 
affected by the categories which are used. Where categories are combined (for 
example, to ensure a minimum of five in each cell) or where categories are 
based on grouped frequencies, the researcher needs to be very aware that: 


e the ‘boundaries’ for the categories are creations of the researcher; 
e use of different combinations or boundaries will affect the chi-square 
statistic. 


The flexibility in terms of categories in the columns and rows, then, turns out 
to be both a strength and a potential weakness of the test. 


How do | see if two groups or categories are different to a significant level? 


Project researchers often wish to compare two sets of data to see if there is a 
significant difference between them. For example, a teacher might wish to 
compare two sets of test scores obtained from students: one set obtained at the 
start of the school year, the other set obtained from the same students on the 
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same test at the end of the school year. Alternatively, he/she might wish to 
compare the results obtained from students in one class with those from 
another. In small-scale research the need to compare two such groups or 
categories in these ways is very common. 

The most appropriate statistical test for this purpose is the t-test. The t-test 
provides the researcher with a statistical test of significance. It uses the means 
of the two sets of data and their standard deviations to arrive at a figure which 
tells the researcher the specific likelihood that any differences between the two 
sets of data are due to chance. 

Researchers using the t-test take the null hypothesis as their starting point. 
They presume that there is no real difference until they can be persuaded 
otherwise by a statistical test which tells them that there is a very strong likeli- 
hood that any variation found between the two sets of data was the result of 
something other than pure chance. As with the chi-share test, researchers 
normally treat differences as ‘real’ (not a fluke) when they find that there is 
a probability of less than 1 in 20, usually expressed as ‘p < 0.05’, that any 
difference between the two sets of data were due to chance. 


Example: The t-test 
We have two sets of marks obtained from two classes who have both taken the 
same test at about the same time. Is there a significant difference between the 
scores obtained by students in class A and the scores obtained by those in 
class B? The independent groups t-test can be used to answer this. 

Class A (%) Class B (%) 

70 55 

60 45 

45 70 

80 75 

55 50 

65 55 

65 40 

50 60 

80 75 

55 50 

65 55 

65 40 

50 60 

25 70 

80 55 

45 

WS) 

70 

55 

50 
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The t-test has two notable strengths which are of particular benefit for small- 
scale research. The t-test works well with small sample sizes (less than 30) - an 
obvious advantage with small-scale research — and the groups do not have to 
be exactly the same size. A data set of 20 items can be compared with a data set 
of 15, for example. 

There are other statistical tests which compare the results from two sets of 
data that will work with ordinal data rather than interval or ratio data. For 
example, the Mann-Whitney U test is a well-known test that works by using the 
rank-ordering of cases in each group. This is why it can work with ordinal data 
such as examination grades (A, B, etc.) if actual percentage scores are not avail- 
able. The Mann-Whitney U test works by producing one overall ranking which 
is made from both groups being investigated. The positions of members of each 
group on this combined ranking are then used as the basis for calculating 
whether there is a statistically significant difference between the two groups. 


How do I see if three or more groups or categories are significantly related? 


This question is likely to be moving beyond the scope of interest for project 
researchers. It suggests a fairly large data set with more than two variables 
potentially connected and worthy of statistical test. It calls for a basic factor 
analysis, such as one-way ANOVA (analysis of variance). This test analyses the 
variation within and between groups or categories of data using a comparison 
of means. 


How do | assess the strength of a relationship between two variables? 


This question is not concerned with the likelihood of any apparent connection 
between two variables being real; it is concerned with how closely two variables 
are connected. The relevant statistics here are correlations. 

Correlations between two variables can be visualized using a scatter plot. 
These need ratio, interval or ordinal data, and cannot be used with nominal 
data. They also require reasonably large data sets to work best (see Figure 13.1). 


FIGURE 13.1 Correlations and scatter plot examples 


7 Perfect positive + 7 Perfect negative 
6 correlation + 6 + correlation 

5 + 5 

4 + 4 

3 + 3 

2 e 2 

1 + 1 

0 0 
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The two most commonly used correlation statistics are Spearman’s rank cor- 
relation coefficient (which works with ordinal data) and Pearson’s product 
moment correlation coefficient (which works with interval and ratio data). 

These statistical tests are used to arrive at a correlation coefficient based on the 
data. The correlation coefficients range as shown in Table 13.7. 


Table 13.7 Correlation coefficients range 


+1 O -1 
perfect no perfect 
positive relationship negative 
correlation correlation 
(one goes up, (one goes up, 
other goes up) other goes down) 


In reality, correlations are unlikely to be perfect. Researchers generally 
regard any correlation coefficient between 0.3 and 0.7 (plus or minus) as dem- 
onstrating some reasonable correlation between two variables. 0.3 is reason- 
ably weak, 0.7 is reasonably strong. 


A Caution: correlation does not indicate cause 


It is important to appreciate that correlation is different from cause. 
The connection between two variables that might be demonstrated 
using a correlation test says nothing about which variable is the 
dependent and which is the independent one. It says nothing about 
which is the cause and which is the effect. It only establishes that there 
is a connection, with a specified closeness of fit between the variables. 

Researchers wishing to investigate connections in terms of cause 
and effect need to use regression analysis. 


Presenting the data — tables and charts 


Statistical packages and spreadsheets generally contain a table-making and a 
chart-making facility. So ‘drawing’ the tables and charts does not generally 
pose much of a problem for project researchers. The skill of producing good 
tables and charts tends to revolve far more around ‘design’ issues and the 
researcher’s ability to do the following: 
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e present enough information without ‘drowning’ the reader with information 
overload; 

e help the reader to interpret the table or chart through visual clues and 
appropriate presentation; 

e use an appropriate type of table or chart for the purpose at hand. 


Vital information 


There are certain pieces of information which must always be included. If 
these are absent, then the table or chart ceases to have much value as far as the 
reader is concerned. The table or chart must always have: 


e atitle; 

e information about the units being represented in the columns of the table or on 
the axes of the chart (sometimes this is placed by the axes, sometimes by the 
bars or lines, sometimes in a legend); 

e the source of the data, if they were originally produced elsewhere. 


Added to this, in most cases — though not all: 


e the horizontal axis is the ‘x axis’ and is used for the independent variable; 
e the vertical axis is the ‘y axis’ and is used for the dependent variable. 


There are some exceptions to the rule about the x axis being the horizontal 
axis, and these are mentioned below. When constructing a chart or table, 
though, it is important for the researcher to have a clear view about which 
variable is seen to be the ‘cause’ and which is the ‘effect’. The variable which is 
treated as the ‘cause’ is the independent variable (normally on the horizontal 
axis) and the variable which is being affected is the dependent variable (nor- 
mally on the vertical axis). Dates (for example, 1980, 1990, 2000, 2010) are 
usually on the horizontal axis and are treated as an independent variable. 

If the researcher pays attention to these points, the remaining dangers to the 
success of the table or chart generally come from trying to be too ambitious 
and putting in too much detail. Simplicity is a virtue that is hard to overstress 
in relation to the production of tables and charts. If too much data are supplied 
in a table, it becomes almost impossible for the reader to decipher the meaning 
of the data. If too many frequencies are presented in a chart, it becomes dif- 
ficult to grasp the key points. Too many patterns, too many words, too many 
figures, and the impact of the table or chart is lost. 


Good practice: the presentation of tables and charts 
As far as tables and charts are concerned, simplicity is a virtue. 
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Tables 


One of the main virtues of tables is their flexibility. They can be used with 
just about all types of numerical data. Computer software offers a variety of 
templates for tables, which aid the presentation of data in a visually attractive 
fashion. 

The degree of complexity of the table depends on two things: the audience 
being addressed and the restrictions imposed by the format of the document 
in which the table appears. For the purposes of most project researchers it is 
likely that tables will remain fairly simple in structure. If the table does need 
to involve a considerable number of rows, the researcher should be prepared to 
insert horizontal and vertical ‘breaks’ or blank lines to ease reading and 
to avoid the eye slipping to the wrong line of data (see Table 13.8). 


Table 13.8 Table example 
Days absent from work through illness during the previous year, 30 employees 


Days Frequency Days Frequency Days Frequency 


(0) 4 5 2 10 3 
1 1 6 3 12 1 
2 1 7 al 13 1 
3 6 8 2 15 1 
4 2 9 1 36 1 


Source: Human Resources Dept, Healthy Life Insurance Co. 


One particularly common use of tables is to present a comparison of sets of 
nominal data. Such tables are known as contingency tables (see Table 13.9). 
They allow a visual comparison of the data and also act as the basis for 
statistical tests of association, such as the chi-square test. 


Table 13.9 Contingency table example 
Days absent from work through illness during the previous year, 30 employees 


Days lost Male Female 
(n= 14) (n= 16) 

0-5 days 21 18 

6 or more days 74 82 


Source: Human Resources Dept, Healthy Life Insurance Co. 


Bar charts 


Bar charts are an effective way of presenting frequencies, and they are very 
common in reports of small-scale research. They can be used with nominal 
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data and with discrete data. The principle behind them is that the bars should 
be of equal width, with the height of the bars representing the frequency 
or the amount for each separate category. Conventionally, there is a space 
between each bar. Their strength is that they are visually striking and simple to 
read — provided, that is, that not too many categories are used. The fewer the 
categories (bars) the more striking the impact. Beyond ten categories bar charts 
tend to become too crowded and confusing to read. Figure 13.2 shows a simple 
bar chart. 

There are many variations on the standard bar chart. The pictogram embel- 
lishes the basic idea by using symbols or images instead of the standard bar. It 
does not tell you anything more, but it catches the eye more readily. 

Bar charts are sometimes turned on their sides to make it easier to label the 
categories (see Figure 13.3). In such horizontal bar charts, the vertical axis 
becomes the x axis — as an exception to the convention. 

Sometimes each bar incorporates the component elements of the total 
for a category. This is known as a stacked bar chart (see Figure 13.4). Interest- 
ingly, stacked bar charts can be used as an alternative to pie charts (see 
below) because they clearly show the relative proportions of the factors 
that make up the total. They have an advantage, too. A number of stacked 
bars can be incorporated into the same chart, allowing a more direct visual 
contrast than could be achieved by placing a similar number of pie charts 
together. 


FIGURE 13.2 Simple bar chart example 


Days absent from work through illness during the previous year, 30 employees 


Number of employees 


ji 0-2 3-5 6-8 9-11 12-14 15+ 
units 995 days days days days days 
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Source: Human Resources Dept, Healthy Life Insurance Co. 
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FIGURE 13.3 Horizontal bar chart example 


Days absent from work through illness during the previous year, 30 employees 


15+ days 
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Source: Human Resources Dept, Healthy Life Insurance Co. 


FIGURE 13.4 Stacked bar chart example 


Days absent from work through illness during the previous year, 30 employees 


Legend 
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Number of employees 
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Source: Human Resources Dept, Healthy Life Insurance Co. 
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Histograms 


A histogram, like a bar chart, is a valuable aid to presenting data on frequencies 
or amounts (see Figure 13.5). Like a bar chart, the bars are normally of equal 
width and where they are of equal width, the height of the bar is used to depict 
variations in the frequencies or amounts. What distinguishes a histogram 
from a bar chart is that the histogram is used for continuous data, whereas a bar 
chart is used for discrete data or nominal data. In contrast to the bar chart: 


e there are no gaps between the bars; 
e the data ‘flow’ along the x axis, rather than being distinct and separate items. 


FIGURE 13.5 Histogram example 


Days absent from work in the previous year by age* of employees 
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Note: *Age at last birthday 
Source: Human Resources Dept, Healthy Life Insurance Co. 


Scatter plots 


A scatter plot is used to display the extent of a relationship between two variables 
(see Figure 13.6). The data are represented by unconnected points on a grid, 
and the clustering or spread of such points is the basis for detecting a close 
co-variation between two variables, or a not so close co-variation. The closer 
the points come together, the closer the relationship between the variables on 
the x axis and the y axis. The more spread out the points are, the less closely the 
variables are connected. The extent of the relationship is made nicely visible; 
indeed the scatter plot is particularly good at showing patterns and deviations. 
There is a drawback, however. It does need a reasonable amount of data to 
reveal such a pattern. 
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FIGURE 13.6 Scatter plot example 


Absenteeism and length of service 


Days absent in the previous year 
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Years in employment with the company 


Source: Human Resources Dept, Healthy Life Insurance Co. 
Line graphs 


A line graph is used for depicting development or progression in a sequence of 
data (see Figure 13.7). It is good for showing trends in data. The most common 
use of the line graph is with dates. A time sequence is plotted left to right on 


FIGURE 13.7 Line graph example 


Total days absent from work through illness per year (1999-2006), 
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Source: Human Resources Dept, Healthy Life Insurance Co. 
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the x axis. The line varies in height according to the frequency or the amount 
of the units. Line graphs work better with large data sets and big variations. 
A bar chart can be used to do the same thing, and sometimes bars and lines are 
combined within the same chart. 


Pie charts 


Pie charts, as their name suggests, present data as segments of the whole pie 
(see Figure 13.8). Pie charts are visually powerful. They convey simply and 
straightforwardly the proportions of each category which go to make up the 
total. In most cases, the segments are presented in terms of percentages. 

To enhance their impact, pie charts that wish to draw attention to one 
particular component have that segment extracted from the rest of the pie. 
On other occasions, all segments are pulled away from the core in what is 
known as an exploded pie chart. 

Weighed against their visual strength, pie charts can only be used with one 
data set. Their impact is also dependent on not having too many segments. As 
a rule of thumb, a pie chart should have no more than seven segments and 
have no segment which accounts for less than 2 per cent of the total. 


FIGURE 13.8 Pie chart example 


Proportion of employees absent from work through illness during the previous year, 
by total days absent, 30 employees 


Legend 
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Source: Human Resources Dept, Healthy Life Insurance Co. 


As a form of chart they tend to take up quite a lot of space on a page. 
The labels need to be kept horizontal, and this spreads the chart out across the 
page. As good practice, incidentally, the proportions should be shown as a 
figure, where possible within the segments. 
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/\ Caution: avoid distorting the data 


The presentation of quantitative data in graphs and charts can be used 
to mislead and give the wrong impression. Playing around with the 
scales on the x axis and the y axis can artificially exaggerate or under- 
play the appearance of differences in the data. Care needs to be taken 
to avoid distorting the data in this way. 


Good practice: tables and charts, which ones to choose? 


The researcher needs to make certain strategic decisions in relation to the 
choice of which way to represent the quantitative data. It is not a matter of 
some charts being better than others as much as some charts being better 
suited to specific purposes than others. Early on, researchers need to decide 
whether the chart they have in mind is concerned with presenting information: 


On amounts or frequencies In which case tables, bar charts or histograms are 
appropriate 

Showing the extent of a In which case scatter plots or contingency tables are 

relationship appropriate 

Covering a span of time In which case line graphs are appropriate 

About proportions of a total In which case pie charts or stacked bar charts are 


appropriate 


Validating the data 


The analysis of quantitative data should include efforts to ensure that, as far as 
possible: 


e the data have been recorded accurately and precisely; 

e the data are appropriate for the purposes of the investigation (we must feel 
assured that we are measuring the right thing); 

e the explanations derived from the analysis are correct. 


The first step towards validation of the data involves checking the data to make 
sure that they contain no errors arising from mistakes with data entry. This can 
be quite a laborious process, but it is vital (unless the data files are constructed 
automatically). Where the data files have been entered via a manual process, 
with people typing in data from one source to another, there is the prospect of 


268 THE GOOD RESEARCH GUIDE 


errors. It is the duty of the researcher to inspect the data files and check 
them against the sources. With small-scale research this might well involve 
the whole dataset; with larger research it can involve checking a sample. 
For example, a questionnaire survey involving 100 completed questionnaires 
with 15 questions on each would require 1,500 keystrokes to enter the data 
manually. Even an error rate of 0.5 per cent would mean that 7 or 8 responses 
would be wrong. The researcher should take measures to eliminate such errors 
from the final ‘cleansed’ data file. 

To address the three points above there are also a number of checks that can 
be undertaken which are designed to provide assurances about the validity of 
the data. If the data are to be regarded as valid then the researcher needs to 
know that: 


1 The research instrument does not vary in the results it produces on different 
occasions when it is used. In principle this can be checked by using the 
research instrument on a later occasion and then comparing the results. 
The ‘test-retest’ approach should produce results that are very similar, 
thus boosting the researcher’s confidence that the instrument is reliable. 
Alternatively, there are ways of splitting the dataset in half and comparing 
one half with the other in terms of the findings they contain. Again, a 
good level of consistency using this ‘split-half approach serves to validate 
the data. 

2 Respondents provide the same kind of answers to similar questions and generally 
answer in a consistent fashion. There are two main ways in which quantitative 
researchers can address this issue. First, they can look for consistency 
within the data in terms of respondents with similar profiles giving similar 
answers. Overall, the researcher is looking for consistent patterns of 
responses. Second, researchers can use ‘check questions’ that are 
deliberately placed within a questionnaire to see whether people’s 
responses are the same to a very similar question. 

3 Different researchers would arrive at similar conclusions when looking at the same 
data. When researchers are recording data based on observations, there is 
the possibility that individuals might arrive at different conclusions about 
‘what happened’ and thus record different data. To confirm that this is not 
the case, it is possible to check on inter-observer consistency by comparing the 
results recorded by two or more researchers. 

4 The findings will apply to other people and other contexts. The validity of the 
data can be assessed in terms of how far the findings from the research can 
be generalized. This requires comparison with other similar settings to see 
whether the results would apply equally well in different circumstances. 

5 The analysis is ‘correct’. It might be better to say ‘works’ rather than ‘correct’, 
but the underlying point is much the same. The researcher needs to validate 
the research by checking that the analysis is better than the available alter- 
natives. To do this, they need to consider whether competing theories 
might provide a better job of explaining the data than the one offered 
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by the researcher. Serious consideration needs to be given to alternative 
explanations of the research results. 


Advantages of quantitative analysis 


e Scientific. Quantitative data lend themselves to various forms of statistical 
techniques based on the principles of mathematics and probability. Such 
statistics provide the analyses with an aura of scientific respectability. The 
analyses appear to be based on objective laws rather than the values of 
the researcher. 

e Confidence. Statistical tests of significance give researchers additional cred- 
ibility in terms of the interpretations they make and the confidence they 
have in their findings. 

e Measurement. The analysis of quantitative data provides a solid foundation 
for description and analysis. Interpretations and findings are based on 
measured quantities rather than impressions, and these are, at least in 
principle, quantities that can be checked by others for authenticity. 

e Analysis. Large volumes of quantitative data can be analysed relatively 
quickly, provided adequate preparation and planning have occurred in 
advance. Once the procedures are ‘up and running’, researchers can 
interrogate their results relatively quickly. 

e Presentation. Tables and charts provide a succinct and effective way of organ- 
izing quantitative data and communicating the findings to others. Widely 
available computer software aids the design of tables and charts, and takes 
most of the hard labour out of statistical analysis. 


Disadvantages of quantitative analysis 


e Quality of data. The quantitative data are only as good as the methods used 
to collect them and the questions that are asked. 

e Technicist. There is a danger of researchers becoming obsessed with the 
techniques of analysis at the expense of the broader issues underlying the 
research. Particularly with the power of computers at researchers’ fingertips, 
attention can shift from the real purpose of the research towards a blinkered 
concern with the technical aspects of analysis. 

e Data overload. Large volumes of data can be a strength of quantitative analy- 
sis but, without care, it can start to overload the researcher. Too many cases, 
too many variables, too many factors to consider — the analysis can be 
driven towards too much complexity. The researcher can get swamped. 
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e Quantitative analysis is not as scientifically objective as it might seem on the 
surface. Researchers have an element of discretion and choice when con- 
ducting their data analysis, just as they do with qualitative data. And their 
decisions can influence the findings. Although the analysis of quantitative 
data might seem to be technical and scientific, in reality the researcher still 
has the ability to influence the findings in subtle ways. 
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Checklist for the use of basic statistics 


When using basic statistics in your research you should feel confident 
about answering ‘yes’ to the following questions: 


Am | clear about which type(s) of data | am using 
(nominal, ordinal, interval, ratio)? 


Is the dataset sufficiently large to warrant statistical analysis? 


Are any statistical procedures used for data analysis 
suitable for: 


the type of data | have collected (i.e. nominal, 
ordinal, interval, ratio)? 

describing frequencies (e.g. mean, median, 
mode, standard deviation)? 

looking for connections (e.g. correlation, 
chi-square, t-test)? 


Have any statistical tests of significance achieved a 
satisfactorily low probability (p < 0.05) to justify 
conclusions based on the data? 


Are the results of data analysis presented in appropriate 
types of table or chart? 


Do the tables or charts avoid ‘information overload’? 


Are the tables or charts clear and thoughtfully designed? 


Has all necessary information been supplied on the units 
being represented in the columns of the tables or on the 
axes of the charts? 


If the data have been derived from someone else’s 
research, has the original source been acknowledged? 


© M. Denscombe, The Good Research Guide. Open University Press. 
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The principles of qualitative data analysis 


The analysis of qualitative data can take a number of forms, reflecting the 
particular kind of data being used and the particular purposes for which they 
are being studied. There is, therefore, no single approach to the analysis of 
qualitative data that covers all situations. However, there are some general 
principles that are commonly associated with qualitative data analysis and 
these can serve as signposts to the kind of things that project researchers 
should bear in mind when doing qualitative research. Broadly speaking, the 
analysis of qualitative data tends to be regarded as: 


e iterative: rather than analysis being a one-off event taking place at a single 
point in time, the analysis tends to be an evolving process in which the data 
collection and data analysis phases occur alongside each other. 
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e inductive: analysis tends to work from the particular to the general. From the 
detailed study of localized data the analysis attempts to arrive at more 
abstract and generalized statements about the topic. 

e researcher-centred: the values and experiences of the researcher are seen as 
factors influencing the analysis. The researcher’s ‘self-identity’ is treated 
as significant in relation to the analysis. 


Types of qualitative data (words and images) 


Qualitative data take the form of words (spoken or written) and visual images 
(observed or creatively produced). They are associated primarily with strategies 
of research such as case studies, grounded theory, ethnography and phenom- 
enology, and with research methods such as interviews, documents and 
observation. Qualitative data, however, can be produced by other means. For 
example, the use of open-ended questions as part of a survey questionnaire 
can produce answers in the form of text — written words that can be treated as 
qualitative data. The kind of research method used, then, does not provide the 
defining characteristic of qualitative data. It is the nature of the data produced 
that is the crucial issue. It is important to bear this point in mind and to 
recognize that qualitative data can be produced by a variety of research 
methods, as Table 14.1 indicates. 


Table 14.1 Types of qualitative data 


Source of data Research Format of data 
method 
Interview talk Interviews Recorded speech 


Reports, diaries, minutes of meetings 


Scripts (e.g. for political speeches or media programmes) l Documents Printed text 


Interactions between people (including naturally 


occurring actions, responses, language) Photographs 
Events (e.g. ceremonies, rituals, performances) Observation Pictures 
Artefacts, symbols, cultural objects (e.g. paintings, Video recordings 


advertisements) 


Answers to open-ended questions Questionnaires Printed text 
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Preparing qualitative data for analysis 


Qualitative data need to be prepared and organized before they can be ana- 
lysed. It is important to appreciate that in a ‘raw’ condition qualitative data are 
likely to be difficult to interrogate in any systematic and meaningful fashion. 
In all probability it will be difficult for the researcher to compare aspects of the 
data or find recurrent themes. Before the data can be used for research purposes, 
therefore, they need to be collected, processed and filed in a way that makes 
them amenable to analysis. 

First, the original data should be protected. Back-up copies should be made of 
all original materials and it is generally advisable to use these back-up copies 
during the process of data analysis so that the originals can be preserved and 
protected against any unintentional corruption or damage. Any loss or dam- 
age to the original material can be catastrophic for the research. The original 
should be stored safely — preferably in a location quite separate from the 
back-up copies, which now become the ‘working copies’ that the researcher 
uses for analysis. 


Good practice: make back-up copies 

Because qualitative data tend to be irreplaceable it is good practice to make a 
duplicate of any recordings, to make copies of computer files and to photocopy 
any visual images or documents such as field notes or transcripts. 


Second, the data should be catalogued and indexed. Each piece of ‘raw data’ 
should be identified with a unique serial number for reference purposes. The 
importance of this is that when analysing the data it is vital that the researcher 
is able to return to points in the data which are of particular interest. Without 
an adequate reference system, it will be virtually impossible for the researcher 
to navigate back and forth through the data, or to record which bits of data 
are significant. There are various ways of doing this but, to illustrate one possi- 
bility, when indexing page 10 of a transcript of an interview that took place 
with Ms Hallam at Dovedale High School, Ms Hallam might be allocated 
TO7 (the seventh teacher interviewed, perhaps), Dovedale High School might 
be allocated the code 13 (as school number 13), so that the actual code added 
to the transcript would be anonymized as [T07:13: p.10]. 
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Transcribing audio recordings of interviews 


If using audio recordings, they should be transcribed and annotated. Tran- 
scriptions help with detailed searches and comparisons of the data. They are 
also necessary when using qualitative data software (see below). The amount 
of the raw data that needs to be transcribed will depend on the use to which 
the data is being put. If the contents of an interview are being used for the 
factual information they provide, for example, as part of a ‘descriptive account’, 
then the researcher can be quite selective; transcription might only be needed 
for the purposes of small extracts that can be used as ‘quotes’ to illustrate 
particular points when writing up the findings. If the researcher is looking for 
the underlying structure of the talk or the implied meanings of a discussion, 
the audio recordings will need to be transcribed quite extensively — possibly 
in their entirety. 
Se, 
Link up with Interviews, Chapter 10 


There might be a temptation to regard transcribing audio recordings as 
a straightforward process. In practice, researchers soon find that it is not. 
First of all, it is very time-consuming. For every hour of talk on an audio 
recording it will take several more to transcribe it. The actual time will depend 
on the clarity of the recording itself, the availability of transcription equip- 
ment and the typing speed of the researcher. When you are planning interview 
research, it is important to bear this in mind. From the point of view of the 
project researcher, the process of transcribing needs to be recognized as a 
substantial part of the method of interviewing and not to be treated as some 
trivial chore to be tagged on once the real business of interviewing has been 
completed. 

The process of transcription is certainly laborious. However, it is also a very 
valuable part of the research, because it brings the researcher ‘close to the data’. 
The process brings the talk to life again, and is a real asset when it comes to 
using interviews for qualitative data. Added to this, the end-product of the 
process provides the researcher with a form of data that is far easier to analyse 
than the audio recording in its original state. 


Good practice: allow sufficient time for the transcription of audio recordings 
The transcription of audio recordings is a time-consuming activity. For every 
one hour of interview recording it generally takes two to three hours of work to 
transcribe the talk into text. 
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Annotations 


When transcribing an audio recording, the researcher should put informal 
notes and comments alongside the interviewee’s words. These annotations 
can be based on the memories that come flooding back during the process of 
transcribing. They should also draw on field notes taken during the interview 
or notes made soon afterwards about the interview. They should include 
observations about the ambience of the interview and things like gestures, 
outside interferences, uncomfortable silences or other feelings that give a 
richer meaning to the words that were spoken. These annotations should be 
placed in a special column on the page. 


Line numbering and coding 


To locate different parts of the transcript and to help navigate through to 
particular points in it, each line in the transcript should ideally have a unique 
line number by which to identify it. Qualitative data analysis software always 
includes a facility to do this, but it is also easily accomplished using basic word 
processing software such as MS Word. The page layout for the transcript 
should include a reasonable amount of clear space on which to write notes and 
there should be a wide margin on the side of the page that can be used later for 
coding and classifying the data and for writing notes. Qualitative data analysis 
software allows this automatically. 


Problems of transcription 


The transcription of audio recordings is not a straightforward mechanical 
exercise in which the researcher simply writes down the words that are spoken 
by research participants. In practice, it can be quite challenging, and there are 
three main reasons for this: 


e The recorded talk is not always easy to hear. Especially with group interviews, 
but also with one-to-one versions, there can be occasions when more than 
one person speaks at the same time, where outside noises interfere or where 
poor audio quality itself makes transcribing the words very difficult. There 
is a fine dividing line here between the need to ditch these parts of the 
interview records and disregard them as worthwhile data, and the need to 
exercise some reasonable interpretation about what was actually said. 

e People do not always speak in nice finite sentences. Normally, the researcher 
needs to add punctuation and a sentence structure to the talk, so that a 
reader can understand the sequence of words. The talk, in a sense, needs to 
be reconstructed so that it makes sense in a written form. This process takes 
time — and it also means that the raw data get cleaned up a little by the 
researcher so that they can be intelligible to a readership who were not 
present at the time of the recording. 
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e Intonation, emphasis and accents used in speech are hard to depict in a transcript. 
There are established conventions which allow these to be added to a writ- 
ten transcript, but the project researcher is unlikely to be able to devote the 
necessary time to learn these conventions. The result is that transcripts in 
small-scale research are generally based on the words and the words alone, 
with little attempt to show intonation, emphasis and accents. Nor are pauses 
and silences always reported on a transcript. The consequence is that, in 
practice, the data are stripped of some of their meaning. 


Example: interview transcripts 


Part of the transcript of a group interview conducted with 15- and 16-year-olds 
about peer pressure and alcohol consumption. The names have been changed. 

Notice how, typical of a group interview, more than one person is trying to 
talk at once. Elise cuts across James, and then gets interrupted in turn by 
Gordon, who manages to hold the spotlight for a moment or two. When more 
than one person is talking at one time it makes transcription difficult because 
it is hard to capture on tape what is being said. It tends to become something 
of a cacophony. 


Line of the Code for Notes 
transcript the content 

46 Interviewer: Do you feel under pressure to drink when 

47 you go out with your mates because they are drinking? 

48 

49 James looked 
50 James: Yeh, but it’s not always . . . (untranscribable) annoyed as 
51 Elise 

BZ Elise: . . . it's worse with the lads, | think, because. . and Gordon 
5S talked over him 
54 Gordon: . . . | was on holiday, and | don’t like beer, and 

55 all my friends were drinking. | was drinking half a glass 

56 and | thought | can stick that out because | don’t like it 08 

57 at all. | was drinking it and thinking this is horrible, just 

58 to look like one of the crowd, because | felt a bit left Gordon says 
59 out. My brother came up to me in front of the crowd later that he 
60 and said ‘Gordon you don’t like lager!’ So all my drinks beer 
61 friends found out, and they said ‘Look, we understand 15 regularly now. 
62 if you don’t like it’ — and some of the people there were 

63 just like me, they didn’t like it. 

64 


65 
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Key point: transcription is not always simple and straightforward 


Transcription is not a mechanical process of putting audio-recorded 
talk into written sentences. The talk needs to be ‘tidied up’ and edited 
a little to put it in a format on the written page that is understandable 
to the reader. Inevitably, it loses some authenticity through this 
process. 


Computer assisted qualitative data analysis 


A number of computer software packages have been developed specifically 
to help with the analysis of qualitative data. These are generally known as 
CAQDAS which stands for computer assisted qualitative data analysis software. 
Some of these are more popular than others and some are more expensive than 
others. The best-known is NVivo, but there are many others that can be used 
and there are some valuable sites online which review the many alternatives 
(e.g. http://onlineqda.hud.ac.uk/index.php or http://caqdas.soc.surrey.ac.uk/ 
index.html). There are also a number of books that provide detailed guidance 
on the use of software like NVivo (e.g. Bazeley 2007, Di Gregorio and Davidson 
2008, Gibbs 2002, Lewins and Silver 2007). 

It is advisable for the project researcher to use one of these computer soft- 
ware packages because, apart from anything else, they are designed to ensure 
that qualitative data are organized and stored in an appropriate fashion. 
In effect, using one of the packages obliges the researcher to prepare the data 
suitably and to take advantage of the computer’s abilities to manage: 


e the storage of data. Interview transcripts, field notes and documents can be 
kept in text files. Copies are quick and easy to make. 

e the coding of data. The data can be easily coded — in the sense of both indexing 
the data (serial codes) and categorizing chunks of the data as a preliminary 
part of the analysis. 

e the retrieval of data. The search facilities of the software make it easy to locate 
data once it has been coded and to pluck it out for further scrutiny. 


A Caution: computer software and data analysis 


Qualitative researchers should make use of CAQDAS packages because 
they help with the preparation and organization of qualitative data, but 
these CAQDAS packages should not be regarded as a quick and easy 
fix for the task of analysing qualitative data. For one thing, they do not 
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actually do the analysis — the researcher still needs to decide the codes 
and look for the connections within the data. The software helps with 
this — but there is no automatic processing of the data comparable 
with the way that statistics packages analyse quantitative data. And, of 
course, the use of computer software, no matter how straightforward, 
involves some start-up time. For the first-time user, there is the time 
taken in getting to know the package and understand its potential. 
There is also the time taken for inputting the data. 


Approaches to the analysis of qualitative data 


Qualitative data analysis is predominantly concerned with the analysis of 
talk and text. Although the analysis of images and artefacts offers interesting 
opportunities for qualitative researchers, the use of image-based research is 
not particularly widespread and in relation to small-scale social research it 
occupies what might be seen as a niche position. So, while the specific issues 
concerned with the analysis of images and artefacts are covered in this chapter, 
the main focus is on the analysis of talk and text. Through a discussion of 
content analysis, grounded theory, discourse analysis, conversation analysis 
and narrative analysis the chapter outlines the main options available to the 
project researcher and discusses some of the key issues involved. Table 14.2 
summarizes the different purposes of the approaches and how these are 
reflected in their treatment of data. 

Across the range of approaches it is evident that there are different visions of 
the work done by words. The philosophy behind this is fundamental to the 
social sciences but from the position of the project researcher it is perhaps only 
necessary to appreciate that words do a lot more than simply describe some- 
thing. As reflected in the different approaches, words can be analysed in terms 
of the background assumptions needed to interpret the words (what they 
imply and what they presume) or in terms of how words underlie social struc- 
ture and interaction (how they produce things and construct things). So, 
whereas the text and talk might be treated at a surface level for the facts and 
information they contain, it can also be used as a basis for ‘reading between 
the lines’ to see what lies beneath the surface. This can allow the researcher to 
interpret the hidden meanings contained in the talk and text or it can enable 
the researcher to see how things like speeches or stories or conversations are 
structured and sequenced in a particular fashion. 

There is also a difference between the approaches in terms of the extent to 
which bits of the talk and text are used in separation from their context. The 
distinctions here are not hard and fast but the project researcher should be 
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aware of the different levels of contextualization to be found in the approaches. 
The unit of analysis can sometimes be individual words, sometimes sentences 
or paragraphs, other times the whole text, and the difference here revolves 
around how far the different approaches tend to isolate bits of text from (1) the 
words that surround them or from (2) the social context in which the words 
were used. For some kinds of content analysis it is not a problem to focus on 
instances of individual words extracted from the text. For approaches like 
discourse analysis and narrative analysis, it is important not to separate units 
of the text from their wider social context. And with conversation analysis 
there is a particular importance given to the location of the unit of analysis 
within the rest of the text or talk within which it occurs. 

Finally, a word on the underlying philosophies of the various approaches. 
In the case of content analysis, there is a strong focus on measurement and 
it is interesting that although it is used in conjunction with qualitative data, it 
produces quantitative measurements. In this respect it might be seen as quite 
positivistic in its approach. Likewise with grounded theory, the reliance on an 
inductive approach suggests an affiliation with positivism — although as we 
saw in Chapter 6, this is mainly the case with a specific branch of grounded 
theory. Discourse analysis, conversation analysis and narrative analysis, how- 
ever, have little to do with positivism. Their assumptions about social reality 
and their preferences in terms of how the data should be treated are based, 
instead, on elements of post-positivism, critical realism, pragmatism and 
postmodernism. Broadly speaking, they are approaches to the analysis of 
qualitative data that are based on interpretations. 


Content analysis 


Content analysis can be used with any ‘text’, whether it is in the form of 
writing, sounds or pictures. It is used as a way of quantifying the contents of 
that text. Political scientists might use it to study the transcripts of speeches, 
educationists to study the content of children’s books, and historians to study 
statesmen’s correspondence. Whatever its specific application, content analysis 
generally follows a logical and relatively straightforward procedure. 


1 Choose an appropriate sample of texts or images. The criterion for the choice of 
such a sample should be explicit. If the choice involves sampling, then 
there needs to be a justification for the kind of sampling technique used 
(see Chapter 1). 

2 Break the text down into smaller component units. The unit for analysis can be 
each and every word. Alternatively, the analysis can use complete sentences 
as the unit, whole paragraphs or things like headlines. It can also be based 
on visual images or the content of pictures. 
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3 Develop relevant categories for analysing the data. The researcher needs to have 
a clear idea of the kinds of categories, issues and ideas that he or she is 
concerned with and how these might appear in the text. This might take the 
form of ‘key words’ associated with the theme. So, for example, a search for 
sex bias in children’s stories might look for instances of boys’ names and 
girls’ names — the names being treated as indicative of the nature of the con- 
tent. The researcher might also wish to code the text in terms of the kinds of 
names, rather than just how many times such names occur. 

4 Code the units in line with the categories. Meticulous attention to the text is 
needed to code all the relevant words, sentences, etc. These codes are either 
written on the text and subsequently referred to, or entered via a computer 
program specially designed for the purpose. 

5 Count the frequency with which these units occur. The first part of analysis is 
normally a tally of the times when various units occur. 

6 Analyse the text in terms of the frequency of the units and their relationship 
with other units that occur in the text. Once the units have been coded, a more 
sophisticated analysis is possible which links the units and attempts to 
explain when and why they occur in the way they do. 


Ye 
e 


Link up with Documents, Chapter 12 


Content analysis has the potential to disclose many ‘hidden’ aspects of what 
is being communicated through the written text (Table 14.3). The idea is 
that, quite independent of what the writer had consciously intended, the text 
carries some clues about a deeper rooted and possibly unintentional message 
that is actually being communicated. You do not have to base the analysis 
on what the author thought he/she was saying when the text contains more 
tangible evidence about its message (Gerbner et al. 1969; Krippendorf 2004). 
The main strength of content analysis is that it provides a means of quanti- 
fying the contents of a text, and it does so by using a method that is clear and, 


Table 14.3 Content analysis 


Content analysis reveals By measuring 

What the text establishes as relevant What is contained (e.g. particular relevant 
words, ideas) 

The priorities portrayed through the text How frequently it occurs; in what order it 
occurs 

The values conveyed in the text Positive and negative views on things 

How ideas are related Proximity of ideas within the text, logical 


association 
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in principle, repeatable by other researchers. Its main limitation is that it has 
an in-built tendency to dislocate the units and their meaning from the context 
in which they were made, and even the intentions of the writer. And it is 
difficult for content analysis to deal with the meaning of the text in terms of 
its implied meanings, how the meaning draws on what has just been said, what 
follows or even what is left unsaid. In many ways, it is a rather crude instru- 
ment for dealing with the subtle and intricate ways in which a text conveys 
meaning. In practice, therefore, content analysis is at its best when dealing 
with aspects of communication which tend to be more straightforward, obvi- 
ous and simple. The more the text relies on subtle and intricate meanings 
conveyed by the writer or inferred by the reader, the less valuable content 
analysis becomes in revealing the meaning of the text. 


Grounded theory 


The grounded theory approach is primarily associated with the analysis of 
interview transcripts (although it can be used with a variety of types of qualita- 
tive data). The analysis requires a detailed scrutiny of the text and involves a 
gradual process of coding and categorizing the data. The ultimate goal of the 
analysis is to derive concepts and theories that capture the meaning contained 
within the data. The grounded theory approach is ‘inductive’ in the sense that 
it seeks to use findings from particular instances as the basis for developing 
statements that apply at a general level. 


oe 
Link up with Grounded theory, Chapter 6 


Explore the data 


The first task for the researcher is to become thoroughly familiar with the data. 
This means reading and re-reading the transcripts. It entails getting a feel for 
the data and becoming immersed in the minute details of what was said, what 
was done, what was observed and what is portrayed through the data. To start 
with, the researcher can take a fairly superficial look at the text (or images). 
Because some time might have elapsed since the data were collected, and 
because there is likely to be a considerable amount of data to work on, the aim 
should be to refresh the researcher’s memory about the content of the data and 
to remind him/her about the broad scope of the material that is available. 
Returning to the data for further inspection, the researcher should begin to 
cross-reference the material with field notes to enable a better understanding 
of the data in context. Subsequent re-readings of the data should allow the 
researcher to identify themes in the data. 
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Memos 


As the analysis progresses, new things will emerge as relevant or new interpret- 
ations might spring to mind. As new insights come to the researcher, the 
ideas need to be noted. Brief thoughts can be appended to the data but more 
detailed insights are easier kept as memos. These memos are of practical value 
as a means for logging new thoughts and exploring new possibilities in relation 
to the analysis of the data. Such memos, in addition, are valuable in the way 
they provide a documented record of the analytic thinking of the researcher 
as they refine the codes and categories. In this sense memos are a note pad on 
which the researcher records how and why decisions were taken in relation 
to the emerging analysis of the data. They provide a permanent and tangible 
record of the researcher’s decision-making which, in principle at least, other 
researchers could inspect. In effect, they render the process of analysis explicit 
and accountable, and can form part of the ‘audit trail’ (see below). 


Good practice: the use of memos 


Researchers should write memos as a way of keeping track of their decisions in 
relation to the analysis of the data. The memos provide a permanent record 
and can be referred back to as a reminder about the researcher’s thinking at 
specific points during the research. 


Code the data 


Codes are tags or labels that are attached to the ‘raw’ data. They can take the 
form of names, initials or numbers; it does not matter as long as the code is 
succinct and is used systematically to link bits of the data to an idea that relates 
to the analysis. The use of codes in connection with the interpretation of data 
is quite distinct from the process of putting a reference code on the various 
pieces of data so that the parts can be identified and stored in an organized 
manner. During the interpretation of the data the researcher engages in 
analytic coding. 

Whatever the form of the data -interview transcripts, field notes, documents, 
photographs or video recordings — the first thing the researcher needs to do 
is decide on the units that will be used for coding the data. This process is 
sometimes known as ‘unitizing’ the data. With text data the units could be as 
small as individual words, they could be lines in the text, complete sentences 
or they could be paragraphs. Whichever, a decision needs to be made on what 
chunks of the data are to have codes attached to them. 

Next, a decision needs to be made about the kind of thing that will be coded. 
Obviously, this will be guided to a considerable extent by the nature of the 
research problem but, to give an indication of the coding decision involved at 
this point, the researcher could base his/her coding on things like: 
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e a kind of event; 

e a type of action; 

e a shade of opinion; 

e an instance of the use of a particular word or expression; 
e animplied meaning or sentiment. 


Finally, a decision needs to be made on the choice of initial codes that might 
usefully be applied to the data. The researcher can use respondent categories 
for this purpose; that is, base the initial codes on something that springs from 
the contents of the data. Or, the initial codes can be based on personal/profes- 
sional hunches about what will prove useful for the analysis. The choice is 
not crucial at the initial phase of analysis because the codes are subject to a 
continual process of refinement during the research, so if the initial codes are 
‘incorrect’, later versions will be refined and improved. 


Categorize the codes 


The next task is to identify ways in which the codes can be grouped into 
categories. The categories act as an umbrella term under which a number of 
individual codes can be placed. The terms ‘taxonomy’ and ‘typology’ are some- 
times applied to this stage in the analysis, reflecting the general idea of classify- 
ing the various components of the data under key headings. The task for the 
researcher is to ‘make the link’. 


Reduce the number of codes and categories 


At the beginning there are likely to be a large number of codes and categories — 
too many to be useful for any meaningful analysis. Part of the analysis, then, 
is to identify where there is sufficient congruence between them to allow 
some to be merged, and others to be brought together within a broader cat- 
egory. Each of the codes brought together within the category will have 
some feature that is shared with the others and, of course, this feature will be 
particularly relevant for the emerging analysis. At this point, some decisions 
also need to be made about which parts of the data are more important than 
others. Attention has to be focused on these parts, with other parts of the 
data being relegated to the sidelines. The importance of such parts of the data, 
it should be stressed, is determined solely by their significance for the 
emerging analysis. 


Develop a hierarchy of codes and categories 


Making sense of the data will involve differentiating among higher level and 
lower level codes. The higher level codes will be broader and more inclusive. 
Crucially, though, the task for the researcher is to develop hierarchical pyra- 
mids of codes by subsuming some (lower level) codes under other broader 
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codes (higher level). The categories will necessarily operate at a higher level 
of generality which will allow them to cover a number of lower-level, more 
specific codes (see Figure 14.1). 


Check the emerging codes, categories and concepts with the data 


The steps towards refining the categories and concepts, as has been noted, 
form part of an iterative process. They are steps in the ‘data analysis spiral’ 
which means that each task is likely to be revisited on more than one occasion 
as the codes, categories and concepts get developed and refined. 


Good practice: check the emerging analysis against competing theories and 
alternative explanations 


Researchers should review their analysis in the light of other explanations of 
similar data. Competing theories should be considered, and the analysis 
should be evaluated in relation to the alternatives. Particular attention should 
be paid to any instances within the researcher's own data that might appear to 
contradict the emerging analysis. 


Move towards key concepts 


In principle, grounded theory analysis aims to use the higher level codes and 
categories as the basis for identifying key concepts. (See Frequently Asked 
Questions, ‘What is a concept?’ p. 324.) The development of these concepts is 


FIGURE 14.1 The grounded theory approach to the analysis of qualitative data 


Generalized 
conclusions 


Qualitative data 
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the main purpose of the analysis because the concepts provide some new 
understanding of the data and constitute the foundations for any theory or 
general conclusions to emerge from the research. 


Discourse analysis 


Discourse analysis is an approach to the analysis of qualitative data that 
focuses on the implied meaning of the text or image rather than its explicit 
content. It is based on the premise that words and pictures are used not simply 
to depict reality; they are used instead as a way of creating and sustaining 
reality. Discourse analysis involves a ‘deconstruction’ of the data in order to 
expose the ways in which text or visual images do the work of creating or 
sustaining particular aspects of social life. This approach is familiar within the 
fields of social psychology, sociology and linguistics and is used in relation to a 
wide range of social research areas including, for example, cultural studies, 
marketing (especially advertising), education and feminist studies. 

Discourse analysis stems from the philosophical influence of people like 
Wittgenstein (1968) and Austin (1962) who argued that language should be 
recognized as something more subtle and more creative than a straightforward 
transmitter of factual information. Rather than acting as a representation of 
reality, they argued, language should be treated as an active creator of reality in 
its own right. One of the main tenets of discourse analysis, in fact, is that the 
words that appear in a text are not determined by the thing they are used to 
represent. Words are chosen. And they are chosen with the purpose of having 
some effect on those who read them. The aim of discourse analysis then, is to 
‘unpack’ the text or image in order to reveal: 


e what people are trying to do through the talk, text or image; 
e what background assumptions are needed in order for this to be achieved. 


There have been a number of developments of this line of thought, notable 
among which have been the works of postmodernists such as Foucault and 
Derrida. Underlying the many strands, discourse analysis generally approaches 
the analysis of talk, text and images on the basis that they should never be 
taken ‘at face value’ but, instead, should be investigated to reveal the hidden 
messages that they contain and the kind of thinking that needs to be going on 
in the background - implicit and unspoken — in order for them to work. This 
involves the researcher in a ‘deconstruction’ of the data. The data are analysed 
by taking them apart to reveal how they: 


e create meaning; 
e contain hidden messages; 
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e reflect, generate and reinforce cultural messages; 
e involve the reader as an active, not passive, interpreter of the message’s 
content. 


In line with other approaches to qualitative data analysis, discourse analysis 
likes to keep close to the data. The approach is premised on the idea that back- 
ground assumptions (‘discourses’) are displayed and transmitted through talk, 
text or images. The task of the researcher is to interrogate the data with the aim 
of showing just how the discourse can be evidenced through the particular 
data being studied. Discourse analysis, in this sense, does not differ from the 
kind of qualitative data analysis outlined elsewhere in this chapter to the extent 
that it relies on 


e aclose scrutiny of the text/image; 
e a firm grounding of conclusions within the data. 


It does differ, however, in the extent to which the researcher approaches the 
analysis of the data ‘without presuppositions’. On this point there is a dis- 
tinct contrast with, for example, the grounded theory approach. In principle, 
grounded theory follows a line of inductive reasoning in which there is an 
expectation that the data themselves should provide the source for any inter- 
pretation made by the researcher. There are, it should be said, variations within 
the grounded theory camp concerning the degree to which the data alone 
contain the necessary ingredients for analysis. However, it is still fair to say 
that the grounded theory approach sees the data as the primary and principal 
generator of the concepts and theories emerging from the research. 

Discourse analysis, by contrast, tends to rely on concepts and theories gen- 
erated in separation from the specific instance of talk, text or images that is 
being analysed. This is almost inevitable because discourse analysis, in its 
attempt to reveal the cultural assumptions that need to come into play in 
order to understand the data, looks at what is absent from the text or image as 
well as what is contained. It looks at what is implied as much as what is explicit. 
The question arises: ‘How would you know what was missing if you did not 
have some idea of what should be expected to “fill the gaps” and “speak the 
unspoken”?’ The researcher who undertakes discourse analysis, therefore, 
needs to use prior assumptions to analyse the data to a degree that grounded 
theorists would not find comfortable. They must use existing knowledge 
about society, culture, politics, etc. and analyse the data with certain necessary 
preconceptions about the meanings contained in the data. 

‘Critical discourse analysis’ tends to add a political bite to the approach. 
It explores the ways the hidden, background meanings that are used to 
make sense of a text or image serves the interests of some groups more than 
others. In fact, this approach explores how power in society is achieved 
through discourse and how this is evident in particular instances of text or 
images. In doing so, it relies quite heavily on the use of political, economic and 
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social ideologies as its frame of reference for understanding the meanings 
embodied in the data. 

It should be clear through the discussion so far that there are a range of 
approaches subsumed under the generic heading of discourse analysis. It is 
important to appreciate that these can focus on quite different subject matters 
and that they can use forms of analysis that do not always map neatly on to 
one another (Wooffitt 2005). However, as Potter and Wetherell (1994) note, 
they share a particular strength in terms of dealing with the subtlety of qualita- 
tive data. Whether dealing with everyday talk, interview transcripts or adver- 
tising posters, they avoid taking things at face value and allow the nuances of 
the data to be explored in a way that approaches such as content analysis 
cannot. They are, in this respect, quite the opposite of content analysis in which 
the analysis of data focuses purely on the explicit content of the data — the 
words and pictures being taken at face value. 

A disadvantage of using discourse analysis, on the other hand, is that it does 
not lend itself to the kind of audit trail (see below) that might be needed to 
satisfy conventional evaluations of the research. It is not easy to verify its 
methods and findings. This is because the approach places particularly heavy 
reliance on the insights and intuition of the researcher for interpreting the 
data. Some discourse analysts eschew the idea that their research can, or 
should, be subject to any process of verification. They argue, in effect, that the 
findings stand as just ‘the researcher’s interpretation’ — no better and no worse 
than anybody else’s. No more scientific; no more credible. This is a contro- 
versial stance, and it is one that could prove unwise for a project researcher to 
adopt. But, unless the researcher is prepared to risk taking this ‘postmodernist’ 
position, they need to be aware of the difficulties of verification inherent in the 
approach. With this in mind, what practical guidance can be given in relation 
to discourse analysis? 


Be clear about which aspects of discourse are being studied 


The researcher needs to specify whether the particular talk, text or image is 
being used in relation to: 


e the shared background meanings and cultural assumptions necessary to 
make sense of the data themselves (micro); 

e the wider ideology and cultural relations that are created and reinforced 
through the data (macro). 


Be clear about which aspects of deconstruction are best applied 


There is no hard and fast process involved in discourse analysis, but the 
researcher who embarks on deconstructing the data needs to have a clear vision 
of the things he/she is looking for and the way these link with the micro and 
macro aspects of discourse. A number of relevant questions to be asked include: 
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e Is there evidence of a wider ideology being communicated via the talk, text 
and images? 

e Do the data reflect particular social, political or historical conditions in 
which they were produced? 

e In what way does the talk, text or image stem from other sources 
(‘intertexuality’)? 

e How is power or influence being exerted through the talk, text or image? 

e Whose version of reality is being portrayed through this talk, text or image? 

e How are people, processes and objects categorized and defined through the 
talk, text or image? 

e Are there people, processes or objects that are notably absent or excluded? 

e Are there contradictions within the talk, text or image? How are these coped 
with? 

e What things are missing from the talk, text or image? 

e Are there views that appear to be suppressed? 

e Are there things that are not said that might have been expected to be 
stated? 

e What rhetorical or linguistic devices are being used to argue the point? 

e What version of things is being portrayed as normal/abnormal, legitimate/ 
illegitimate, moral/immoral, natural/unnatural, etc.? 

e Would alternative interpretations of the talk, text or image be possible or 
likely depending on the particular audience? 


Conversation analysis 


Conversation analysis is often considered to be a variant of discourse analysis 
because it focuses attention on how things get done through language use. In 
this instance, researchers look at naturally occurring talk and the way that 
routine aspects of everyday life get accomplished through language. There 
have been studies of things like taking turns in everyday conversation, greet- 
ing people, starting a school lesson — apparently mundane and minute parts of 
social life, but ones that advocates of ethnomethodology stress are the funda- 
mental building blocks of social interaction — without which there would be 
no such thing as social life. The structure and sequencing of the talk are crucial 
from this perspective. 

Conversation analysts are inclined to underplay the extent to which they 
draw on presuppositions. However, their analysis must still rely on cultural 
insights. Indeed, the researcher’s background knowledge can be argued to be 
absolutely essential to the sensible interpretation of routine conversation. For 
any reasonable interpretation, it is vital that the researcher has appropriate 
background knowledge, shared with those doing the talking. Otherwise, they 
would literally have no way of knowing what was going on. 
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Narrative analysis 


A narrative relates to a story. A story can be told in many forms; in writing 
through books and in speech through interviews, as well as a whole host of 
other mediums perhaps less commonly used as sources of data by social 
researchers such as visual images, music and drama. Treated as a narrative, 
though, the story needs to have certain properties (Reissman 2003, 2008; Elliott 
2005: Andrews et al. 2008). Normally, it would be expected to do the following: 


e have some specific purpose. This could be a moral message, or it could be an 
account of personal circumstances. Whatever the subject matter of the 
story, it has to be leading somewhere. 

e contain a plot line linking the past to the present. There is normally some sense 
of development and change over time in which past events are recounted 
and linked explicitly to the present. 

e involve people. The theme needs to have a human element to it, referring 
to feelings and experiences in the context of social events and human 
interaction. 


For qualitative research, such stories can be useful in two ways. First, they can 
be analysed in terms of how they construct the social world. The point here is 
that stories can have a special importance in culture which can go well beyond 
that of entertainment. They might be parables with strong moral lessons 
involved. They might be contemporary news stories built on underlying 
beliefs about rights and wrongs in politics. Or they might be ‘classics’ with 
great symbolic significance, handed down from generation to generation as 
part of the transmission of culture, telling of how things were and why things 
happened and serving to bolster the beliefs and values that bind a society 
together. Second, they can be analysed in terms of how they construct a personal 
world. This aspect of narrative analysis links quite closely with research on life 
histories and oral histories. It is concerned with the way that stories are used by 
people as ways of representing themselves and their lives to others. The stories 
are used, consciously or not, as a device through which people describe and 
explain themselves and the circumstances surrounding their existence. 
Narrative analysis applied to any such stories is not overly concerned with 
whether the content of the story is built on truth or myth; the focus is more on 
how those stories work. The stories are analysed to reveal the way the story 
constructs and interprets the social world, and the focus of attention is on: 


e the meanings and ideology the story conveys; 

e the techniques and communicative devices the story-teller uses; 

e how the story links with the cultural and historical context within which 
it is told. 
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The analysis of image-based data 


The kind of analysis used for image-based data depends on the purposes for 
which the data were collected. Broadly, images are used for either, or both, of 
two reasons: 


e for the factual information they contain; 
e for the cultural significance and symbolic meaning that lie behind their 
content. 


There are various strands to the analysis of visual images. Without becoming 
embroiled in the detail of the differences, it is perhaps possible to identify 
three elements to the analysis of image-based data: 


e the image itself: contents, genre, styles; 

e the producer: intentions and context (by whom, when, under what circum- 
stances, why, the intention of the creator); 

e the viewer: interpretation and context. 


The image itself can be valuable as a source of factual information. There is a 
case to be made that images provide a relatively reliable source of data — prefer- 
able, at least, argues Collier (2001: 59), to the ‘deceptive world of words’. 
Objectively frozen in time, the image contains evidence of things with a clarity 
that words could never hope to match. Photographs from archives, film from 
distant fieldwork settings, snapshots from a family album, each contains a firm 
record of events that can be subsequently analysed by the researcher. ‘Such 
images are produced to serve as records of reality, as documentary evidence of 
the people, places, things, actions and events they depict. Their analysis is a 
matter of extracting just that kind of information from them’ (van Leeuwen 
and Jewitt 2001: 4). 


p9 


208 


Link up with Image-based research, pp. 227-8 


To help get to grips with the information contained in large collections 
of images researchers can use content analysis, and it is quite possible to pro- 
duce quantitative data as part of the analysis of images (see Bell 2000, for a 
detailed account of this). In practice, however, image-based research has 
largely favoured a qualitative approach. The key reason for this is that social 
researchers in recent years have become reluctant to take the content of 
images entirely ‘at face value’. Rather than accepting images as neutral depic- 
tions of reality (along with the positivistic connotations this has), most social 
researchers in the twenty-first century would acknowledge, as Rose (2007) 


QUALITATIVE DATA 293 


argues, that images do not contain ‘one right meaning’. What they depict 
is open to interpretation and it would be rather naive to presume that the 
contents of visual images are self-explanatory. Without entirely abandoning 
the use of images in terms of their ostensive content (the ‘facts’ they portray), 
researchers have become more inclined to treat the contents as something that 
needs to be interpreted. The image is not created ‘literally’ but is interpreted. 
The meaning of the image is the thing of value for the researcher, the meaning 
as given within a particular culture, within a given time, within a social 
context within which it was produced. 

Another way of interpreting an image is as a cultural artefact. As a cultural 
artefact, the focus moves away from just the content of the image and what 
this can tell the researcher, to incorporate, also, questions about the context 
in which the image was produced. What purpose did/does it serve? Why was 
it produced? As a source of evidence, the value of the image is in what lies 
behind the surface appearance. To give an illustration, a photograph might 
appear to show a family on a seaside beach. At one level the researcher could 
use the image to get factual information about the style of beachwear worn 
at the time the photograph was taken. Provided the photograph was authen- 
tic, the image acts as a source of factual data. However, that same image 
might be interpreted as a cultural artefact by asking questions about how 
and why it was produced. Who took the photograph? Why was it taken? 
What importance is attached to it that might explain why it has been kept 
over a long period of time? What equipment was used to take the photo- 
graph? Such lines of enquiry could help researchers to understand the sig- 
nificance of family holidays, the role of photo albums and the nature of 
leisure activity in their cultural context. The meaning of the photograph, 
that is, goes beyond the superficial content depicting two adults and two 
children at the seaside. 

Many advertisements now ‘play’, quite consciously, with the relationship 
between the three aspects. They ‘invite’ the viewer to do the following: 


e look at the content at face value; 
e acknowledge the producer’s message (to buy); 
e relate the image and message to themselves, their own wants and needs. 


Another approach to the analysis of images interprets them in terms of what 
they denote. It probes the image as a symbolic representation whose interest lies 
not as much in the ‘facts’ contained in the image as in the significance 
attached to the contents by those who view it. Images, used in this way, pro- 
vide a source for uncovering ideologies and cultural codes. Drawing on the 
field of ‘semiotics’, the researcher first considers the various aspects of the 
content to see what they denote. Taken at face value, what do they ‘say’? 
Next the researcher considers the connotations of the key elements in the 
image. What associations spring to mind when looking at those elements? 
Then, in the final stage of analysis, semiotics moves towards drawing contrasts 
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and similarities with other signs in a way that helps us to understand the 
symbolism of the image. (See Penn 2000, for further information on the 
semiotic analysis of still images.) 

It is important to note that reading the message is not entirely an indi- 
vidual, personal thing in which each viewer interprets things differently. 
While there is certainly some scope for variation, semiotics and cultural stud- 
ies operate on the premise that there is a degree of consensus among viewers 
about the meaning of an image. People who come from a similar cultural 
background, who are members of the same society or who have been brought 
up within a particular ideological tradition (e.g. democracy, Christianity) 
might be expected to make sense of images in roughly the same way. As 
members of social groups they will have been socialized into ways of seeing 
what is meant by visual images - learning from others since their early 
years how to make sense of visual clues contained in things like signs, advert- 
isements, portraits and music videos. We learn not to treat images simply 
in terms of their ‘literal’ contents. Road signs that point upwards are not inter- 
preted to mean ‘go upwards’. 


/\ Caution: context is important 


When using images for research purposes there is always the danger 
that they might be ‘taken out of context’. Events prior and subsequent 
to the moment captured by the image might have a crucial bearing on 
the meaning that can be attributed to it. 


Presenting the data 


An explanation of statistical techniques and data manipulation always 
accompanies the use of quantitative data in research yet, all too often, an 
account of the comparable processes is missing or very scant when it comes to 
reports that have used qualitative data. The process that leads from the data to 
the researcher’s conclusions gets ignored, with the reader of the research report 
being given little or no indication of how the data were actually analysed. The 
quality of qualitative research is increasingly coming under the microscope in 
this respect (Seale et al. 1999, 2004; Silverman 2006) and, quite rightly, those 
who use qualitative data are now expected to include in their accounts of 
research a description of the processes they used to move from the raw data to 
their findings. 
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Good practice: qualitative data analysis should be detailed and rigorous 


Researchers need to approach the analysis of qualitative data with attention to 
detail and rigour. Not only that, they need to be explicit about their analytic 
procedures when writing reports based on their findings. It is not acceptable to 
say ‘this is how | collected the data’ (methods) and ‘these are my results’ 
(findings) without giving an account of how the data were analysed. Just as 
quantitative researchers are expected to explain which statistical procedures 
were used in order to arrive at their conclusions, so qualitative researchers also 
need to explain the processes by which they move from the raw data to their 
eventual findings. 


Qualitative researchers, however, face a special challenge when it comes to 
depicting the process involved in the analysis of their data (Woods 2006). 
Partly, this is because the process itself can be quite complicated. As Marshall 
and Rossman (1999: 150) put it: 


[Qualitative] data analysis is the process of bringing order, structure, and 
interpretation to the mass of collected data. It is a messy, ambiguous, 
time-consuming, creative, and fascinating process. It does not proceed in 
a linear fashion: it is not neat. 


As well as this, the data themselves are not as amenable to being presented in a 
concise fashion as quantitative data. Quantitative data can be represented 
using statistics and tables. These provide a succinct and well-established means 
enabling researchers to encapsulate the findings from large volumes of data 
and provide the reader with explicit information about the techniques used 
for data analysis. The challenge for qualitative researchers, on the other 
hand, is to depict their data within the confines of a relatively few pages when 
dealing with: 


e a fairly complicated and messy process of data analysis; 
e arelatively large volume of data based on words or images. 


To deal with this challenge, qualitative researchers need to come to terms 
very quickly with the fact that it is not feasible for them to present all of their 
data. They need to be selective in what they present and acknowledge that 
they have to act as an editor — identifying key parts of the data/analysis and 
prioritizing certain parts over others. 

Next, and linked with the editorial role, it needs to be recognized that the 
data chosen for inclusion within a report does not constitute evidence that 
proves a point. It is more likely to be introduced as an illustration of a point 
that is being made. Because the reader of the report does not normally have 
access to all the data, they have no way of knowing whether the selection was a 
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fair reflection of the data or not. Further discussion of this point can be found 
in relation to the use of extracts from interview transcripts. 

Finally, the representation of qualitative data/analysis involves an element 
of ‘rhetoric’. That is, the credibility of the research owes something to the skill 
of the researcher in managing to produce a persuasive account of the research 
that ‘touches the right buttons’ and conveys a sense of authority and authen- 
ticity. There are some different styles of writing that qualitative researchers can 
employ to achieve this (van Maanen 1995; Woods 2006). Without delving into 
the rhetorical devices themselves, the point to note is that the credibility of 
qualitative research, although it still depends on providing relevant factual 
details relating to the research procedures, also relies on the literary skills of 
the researcher to write a convincing account. This applies more to qualitative 
research than it does to quantitative research. 


The use of interview extracts in research reports 


Extracts from transcripts play an important role in qualitative research. They 
can be interesting in their own right, giving the reader a flavour of the data and 
letting the reader ‘hear’ the points as stated by the informants. Or they can be 
used as a piece of evidence supporting the argument that is being constructed 
in the report by the researcher. What an extract is very unlikely to do, however, 
is provide proof of a point. There are two reasons for this: 


e The significance of extracts from transcripts is always limited by the fact that they 
are, to some extent, presented out of context. Extracts, as the very word itself 
suggests, are pieces of the data that are plucked from their context within 
the rest of the recorded interview. This opens up the possibility that the 
quote could be taken ‘out of context’. Everyone knows the danger of being 
quoted out of context. The meaning of the words is changed by the fact that 
they are not linked to what came before and what was said after. The shrewd 
researcher will try to explain to the reader the context within which the 
extract arose but, inevitably, there is limited opportunity to do this in 
research reports. 

e The process of selecting extracts involves a level of judgement and discretion on 
the part of the researcher. The selection of which parts of the transcript to 
include is entirely at the discretion of the researcher, and this inevitably 
limits the significance which can be attached to any one extract. It is an 
editorial decision which reflects the needs of the particular research report. 
How does the reader know that it was a fair selection, representative of the 
overall picture? 


As a result, in the vast majority of reports produced by project researchers, 
extracts will serve as illustrations of a point and supporting evidence for an 
argument — nothing more and nothing less. It is important that the project 
researcher is aware of this when using extracts, and avoids the temptation to 
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present quotes from interviewees as though they stand in their own right as 
unequivocal proof of the point being discussed. 

Having made this point, there are some things the researcher can do to get 
the best out of the extracts that are selected. 


e Use quotes and extracts verbatim. Despite the points above about the dif- 
ficulty of transcribing recordings, the researcher should be as literal as 
possible when quoting people. Use the exact words. 

e Change the names to ensure anonymity (unless you have their explicit 
permission to reveal the person’s identity). 

e Provide some details about the person you are quoting, without endanger- 
ing their anonymity. The general best advice here is to provide sufficient 
detail to distinguish informants from one another and to provide the reader 
with some idea of relevant background factors associated with the person, 
but to protect the identity of the person you quote by restricting how 
much information is given (e.g. teacher A, female, school 4, geography, 
mid-career). 

e Try to give some indication of the context in which the quotation arose. 
Within the confines of a research report, try to provide some indication of 
the context of the extract so that the meaning as intended comes through. As 
far as it is feasible, the researcher should address the inevitable problem of 
taking the extract ‘out of context’ by giving the reader some guidance on 
the background within which the statement arose. 


Good practice: the use of diagrams 


In reports based on qualitative research it is particularly useful to include 
diagrams that show the link between the concepts that are developed. This 
provides the reader with a visual picture of what can often be complex links. It 
also gives some transparency to the process of data analysis. 


Verifying the data 


The credibility of research is something that needs to be demonstrated as part 
and parcel of the research process itself. It should not be taken for granted. 
Researchers should not assume that people are naive and will take their find- 
ings at face value. Nor should they count on people to trust the researcher’s 
word on the quality of the findings as an act of faith. For the research to 
achieve credibility it needs to demonstrate in some way or other that the 
findings are based on practices that are acknowledged to be the bases of good 
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research. And, as Silverman (2006, 2010) has stressed, this applies as much to 
qualitative research as it does to quantitative research. 
Conventionally, the bases for judging the credibility of research have been: 


e Validity. This refers to the accuracy and precision of the data. It also concerns 
appropriateness of the data in terms of the research question being investi- 
gated. The basic question is ‘Are the data the right kind for investigating the 
topic and have they been measured correctly?’ 

e Reliability. This refers to whether a research instrument is neutral in its effect 
and consistent across multiple occasions of its use. This is frequently trans- 
lated as the question ‘Would the research instrument produce the same 
results on different occasions (all other things being equal)? 

e Generalizability (external validity). This refers to the prospect of applying the 
findings from research to other examples of the phenomenon. It concerns 
the ability of research findings to explain, or occur in, similar phenomena at 
a general or universal level rather than being something that is unique to 
the particular case(s) used for the research. 

e Objectivity. This refers to the absence of bias in the research. It denotes 
research that is impartial and neutral in terms of the researcher’s influence 
on its outcome, and it denotes processes of data collection and analysis that 
are fair and even-handed. 


The credibility of qualitative research is not easily judged using these criteria. It 
is not feasible, for instance, to check the quality of research and its findings by 
replicating the research in the same way that scientists might repeat an experi- 
ment. The first reason for this is that it is virtually impossible to replicate a social 
setting. Time inevitably changes things and the prospects of assembling equiva- 
lent people in similar settings in a social environment that has not changed are, 
to say the least, slim. The second reason is that the researcher tends to be intim- 
ately involved in the collection and analysis of qualitative data, so closely 
involved that the prospects of some other researcher being able to produce 
identical data and arrive at identical conclusions are equally slim. 

Recognizing such difficulties, some qualitative researchers have suggested 
abandoning any commitment to these conventional ways of judging the 
quality of research. The majority, though, have been rather more reticent to 
abandon the conventional criteria used by the positivistic research. They make 
the point that, although the nature of qualitative research means that it 
will never be possible to be verified in the same way as quantitative research, 
there is still a need to address the need for verification (e.g. Kirk and Miller 
1986; Bryman and Burgess 1994; Miles and Huberman 1994; Seale et al. 1999; 
Silverman 2006). Such qualitative researchers have adopted a more pragmatic 
or ‘subtle’ realist perspective ‘that does not give up on scientific aims as con- 
ventionally conceived, but also draws on the insights of post-scientific concep- 
tions of social research’ (Seale et al. 1999: x). The discussion and guidance that 
follow are rooted in this point of view. 
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Credibility (validity) 


The issue here is the extent to which qualitative researchers can demonstrate 
that their data are accurate and appropriate. Lincoln and Guba (1985) make the 
point that it is not possible for qualitative researchers to prove in any absolute 
way that they have ‘got it right’. (As a consequence, they prefer to use the term 
‘credibility’ in relation to this aspect of the verification of research.) There are, 
though, steps to be taken that can help with the task of persuading readers of 
the research that the data are reasonably likely to be accurate and appropriate. 
These steps provide no guarantee, because none are available. They offer, 
instead, reassurances that the qualitative data have been produced and checked 
in accord with good practice. It is on this basis that judgements can be made 
about the credibility of the data. 

To address the matters of accuracy and appropriateness of data qualitative 
researchers can use: 


e Triangulation. Avoiding any naive use of triangulation, the researcher can 
use contrasting data sources to bolster confidence that the data are ‘on the 
right lines’. 


Link up with Triangulation, Appendix 3 e 
"ee 


e Respondent validation. The researcher can return to the participants with 
the data and findings as a means of checking the validity of the findings. 
This allows a check on factual accuracy and allows the researcher’s under- 
standings to be confirmed (or amended) by those whose opinions, views 
or experiences are being studied. There are, though, some limitations to 
respondent validation, notably in the sense that the analysis of the data 
might take the explanation beyond something that would be immediately 
recognizable to the respondent. 

e Grounded data. One of the key benefits of qualitative research is that the 
findings will have been grounded extensively in fieldwork and empirical 
data. Qualitative research tends to involve relatively long times spent ‘on 
location’ conducting the fieldwork, and it builds on a detailed scrutiny of 
the text or visual images involved. This provides a solid foundation for the 
conclusions based on the data and adds to the credibility of the research. 


Dependability (reliability) 


With qualitative research the researcher’s ‘self’ tends to be very closely bound 
up with the research instrument — sometimes an integral part of it. As a partici- 
pant observer or interviewer, for example, the researcher becomes almost an 
integral part of the data collecting technique. As a result, the question of reli- 
ability translates from ‘Would the research instrument produce the same 
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results when used by different researchers (all other things being equal)?’ to 
‘If someone else did the research, would he or she have got the same results 
and arrived at the same conclusions?’ In an absolute sense there is probably 
no way of knowing this for certain (as indicated above). Yet there are ways 
of dealing with this issue in qualitative research — ways that accord broadly 
with what Lincoln and Guba (1985) call ‘dependability’. Principally, these 
revolve around the demonstration that their research reflects procedures and 
decisions that other researchers can ‘see’ and evaluate in terms of how far 
they constitute reputable procedures and reasonable decisions. This acts as a proxy 
for being able to replicate research. Only if such information is supplied is it 
possible to reach conclusions about how far another researcher would have 
come up with comparable findings. As a check on reliability, this calls for an 
explicit account of the methods, analysis and decision-making and ‘the provi- 
sion of a fully reflexive account of procedures and methods, showing the 
readers in as much detail as possible the lines of enquiry that led to particular 
conclusions’ (Seale et al. 1999: 157). In effect, the research process must be 
open for audit. 


Good practice: an audit trail 


An audit trail should be constructed and mapped out for the reader — allowing 
them to follow the path and key decisions taken by the researcher from con- 
ception of the research through to the findings and conclusions derived from 
the research. But, as Lincoln and Guba (1985: 319) stress, ‘An inquiry audit 
cannot be conducted without a residue of records stemming from the inquiry, 
just as a fiscal audit cannot be conducted without a residue of records from the 
business transactions involved.’ This is why it is good practice to keep a fairly 
detailed record of the process of the research decisions. The principle behind 
the audit trail is that research procedures and decision-making could be 
checked by other researchers who would be in a position to confirm the exist- 
ence of the data and evaluate the decisions made in relation to the data 
collection and analysis. 


Transferability (generalizability) 


Qualitative research tends to be based on the intensive study of a relatively 
small number of cases. Inevitably, this raises questions about how representa- 
tive those cases are and how likely it is that what was found in the few cases 
will be found elsewhere in similar cases. The thorny issue is often raised 
through the question; ‘How can you generalize on the basis of such a small 
number?’ Statistically speaking, this is a reasonable question. It is one that is 
difficult to dodge. 

Some qualitative researchers try to side-step the issue by asserting that it is 
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not their business to make such generalizations. They argue that the findings 
from things like case studies are worthwhile in their own right simply as a 
depiction of the specific, possibly unique, situation. This, however, is a minor- 
ity position. The majority of qualitative researchers accept that the issue is 
relevant but argue that it needs to be approached in a different way when being 
used in relation to qualitative research. They point out that generalizability is 
based on the statistical probability of some aspect of the data recurring else- 
where, and it is a probability that relies on a large sample that is representative 
of the wider population. Research based on small numbers and qualitative data 
needs an alternative way of addressing the issue. This alternative way is what 
Lincoln and Guba (1985) have called ‘transferability’. This is an imaginative 
process in which the reader of the research uses information about the particu- 
lar instance that has been studied to arrive at a judgement about how far it 
would apply to other comparable instances. The question becomes ‘To what 
extent could the findings be transferred to other instances?’ rather than ‘To 
what extent are the findings likely to exist in other instances?’ 


Good practice: information for comparisons 


The researcher needs to supply information enabling others to infer the rele- 
vance and applicability of the findings (to other people, settings, case studies, 
organizations, etc.). In other words, to gauge how far the findings are transfer- 
able, the reader needs to be presented with relevant details on which to base a 
comparison. If, for example, the research is on the eating habits of young 
people and it is based on interviews with 55 students in one school, the 
findings might not be generalizable to the population of students in all 
schools. However, armed with appropriate information about the age/sex/ 
ethnicity of the students and the social context of the school, it becomes 
possible to think about how the findings might apply to similar students in 
similar schools. Armed with the appropriate information, the reader can 
consider the transferability of the findings. 


Confirmability (objectivity) 


This issue of objectivity concerns the extent to which qualitative research can 
produce findings that are free from the influence of the researcher(s) who 
conducted the enquiry. At a fundamental level, it needs to be recognized 
straight away that no research is ever free from the influence of those who 
conduct it. Qualitative data, whether text or images, are always the product 
of a process of interpretation. The data do not exist ‘out there’ waiting to 
be discovered (as might be assumed if a positivistic approach were adopted) 
but are produced by the way they are interpreted and used by researchers. This 
has consequences for the prospects of objectivity. First, it raises questions 
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about the involvement of the researcher’s ‘self’ in the interpretation of the 
data and, second, it raises questions about the prospects of keeping an open 
mind and being willing to consider alternative and competing explanations 
of the data. 


The researcher’s ‘self’ 


The researcher’s identity, values and beliefs cannot be entirely eliminated 
from the process of analysing qualitative data. There are, though, two ways in 
which qualitative researchers can deal with this involvement of the self. On 
the one hand, they can deal with it by saying: 


The researcher’s identity, values and beliefs play a role in the production 
and analysis of qualitative data and therefore researchers should be on 
their guard to distance themselves from their normal, everyday beliefs and 
to suspend judgements on social issues for the duration of their research. 


At this end of the scale, researchers know that their self is intertwined with 
their research activity, but proceed on the basis that they can exercise sufficient 
control over their normal attitudes to allow them to operate in a detached 
manner, so that their investigation is not clouded by personal prejudices. In 
this case, the researcher needs to suspend personal values and beliefs for the 
purposes of the production and analysis of the data. 

On the other hand, qualitative researchers could take the position that: 


The researcher’s identity, values and beliefs play a role in the production 
and analysis of qualitative data and therefore researchers should come 
clean about the way their research agenda has been shaped by personal 
experiences and social backgrounds. 


At the extreme, this approach can take the form of celebrating the extent to 
which the self is intertwined with the research process. There are those who 
argue that their self gives them a privileged insight into social issues, so that 
the researcher’s self should not be regarded as a limitation to the research but 
as a crucial resource. Some feminist researchers and some researchers in the 
field of ‘race’ make the case that their identity, values and beliefs actually 
enable the research and should be exploited to the full to get at areas that will 
remain barred to researchers with a different self. So some feminists argue that 
only a woman can truly grasp the significance of factors concerned with the 
subordination of women in society. Some black researchers would make a 
similar point in relation to ‘race’ inequality. 

Between the two positions outlined above lie a variety of shades of opinion 
about the degree to which the researcher’s self can and should be acknow- 
ledged as affecting the production and analysis of qualitative data. However, 
there is a general consensus on the key point: the role of the self in qualitative 
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research is important. To ignore the role of the researcher’s self in qualitative 
research, then, would be ill-advised. 


A reflexive account of researcher’s self 


The analysis of qualitative data calls for a reflexive account by the researcher 
concerning the researcher’s self and its impact on the research. There is 
a growing acceptance among those involved in qualitative data analysis 
that some biographical details about the researcher warrant inclusion as part 
of the analysis, thus allowing the writer to explore the ways in which he 
or she feels personal experiences and values might influence matters. The 
reader of the research, by the same token, is given valuable information 
on which to base a judgement about how reasonable the writer’s claims are 
with regard to the detachment or involvement of self-identity, values and 
beliefs. 


An open mind 


Qualitative researchers, like all researchers, need to approach the analysis of 
the data with an open mind. If they do not, they can be accused of lacking 
objectivity. In practice, a way of demonstrating such openness is for the 
researcher to take seriously the possibility of having ‘got it wrong’. This is 
actually a fundamental point in relation to the production of knowledge. 
It is linked to the argument that good research is concerned not with the 
verification of findings but with their ‘falsification’. For the project researcher 
there are perhaps two aspects of this which can be addressed quite practically. 
Qualitative researchers should: 


e Avoid neglecting data that do not fit the analysis. Qualitative researchers such 
as Seale et al. (1999) and Silverman (2006) have argued that good research 
should not ignore those instances within the data that might appear to 
disconfirm the researcher’s analysis. There might be only a few such ‘out- 
liers’. There might be just the occasional bit of data that does not fit nicely 
or that contradicts the general trend. These should not be ignored, but 
should be actively investigated to see if there is an explanation for them 
that can be accommodated within the emerging analysis, of whether there 
is some genuine significance in them. Looking for, and accounting for, 
negative instances or deviant cases that contradict the emerging analysis are 
important. 

e Check rival explanations. Alternative possible explanations need to be 
explored. The researcher needs to demonstrate that he or she has not sim- 
ply plumped for the first explanation that fits, rather than seeing if rival 
theories work or whether there are hidden problems with the proposed 
explanation. 
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Good practice: the need to be open about the researcher’s ‘self’ 


Qualitative researchers should provide some biographical details about them- 
selves and their interest in the research. They should also address the issue of 
objectivity by demonstrating the extent to which they have approached the 
research with an ‘open mind’. 


Advantages of qualitative analysis 


There is a richness and detail to the data. The in-depth study of relatively 
focused areas, the tendency towards small-scale research and the generation 
of ‘thick descriptions’ mean that qualitative research scores well in terms of 
the way it deals with complex social situations. It is better able to deal with 
the intricacies of a situation and do justice to the subtleties of social life. 
There is tolerance of ambiguity and contradictions. To the extent that social 
existence involves uncertainty, accounts of that existence ought to be 
able to tolerate ambiguities and contradictions, and qualitative research is 
better able to do this than quantitative research (Maykut and Morehouse 
1994: 34). This is not a reflection of a weak analysis. It is a reflection of the 
social reality being investigated. 

There is the prospect of alternative explanations. Qualitative analysis, because 
it draws on the interpretive skills of the researcher, opens up the possibility 
of more than one explanation being valid. Rather than a presumption 
that there must be, in theory at least, one correct explanation, it allows for 
the possibility that different researchers might reach different conclusions, 
despite using broadly the same methods. 

The data and the analysis are usually ‘grounded’. A particular strength associ- 
ated with qualitative research is that the descriptions and theories such 
research generates are ‘grounded in reality’. This is not to suggest that they 
depict reality in some simplistic sense, as though social reality were ‘out 
there’ waiting to be ‘discovered’. But it does suggest that the data and the 
analysis have their roots in the conditions of social existence. There is little 
scope for ‘armchair theorizing’ or ‘ideas plucked out of thin air’. 


Disadvantages of qualitative analysis 


e The data might be less representative. The flip-side of qualitative research’s 


attention to thick description and the grounded approach is that it becomes 
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more difficult to establish how far the findings from the detailed, in-depth 
study of a small number of instances may be generalized to other similar 
instances. Provided sufficient detail is given about the circumstances of the 
research, however, it is still possible to gauge how far the findings relate to 
other instances, but such generalizability is still more open to doubt than it 
is with well conducted quantitative research. 

e Interpretation is bound up with the ‘self’ of the researcher. Qualitative research 
recognizes more openly than does quantitative research that the researcher’s 
own identity, background and beliefs have a role in the creation of data and 
the analysis of data. The research is ‘self-aware’. This means that the find- 
ings are necessarily more cautious and tentative, because it operates on the 
basic assumption that the findings are a creation of the researcher rather 
than a discovery of fact. Although it may be argued that quantitative 
research is guilty of trying to gloss over the point - which equally well 
applies — the greater exposure of the intrusion of the ‘self’ in qualitative 
research inevitably means more cautious approaches to the findings. 

¢ There is a possibility of decontextualizing the meaning. In the process of coding 
and categorizing the field notes, texts or transcripts there is a possibility that 
the words (or images for that matter) get taken literally out of context. The 
context is an integral part of the qualitative data, and the context refers to 
both events surrounding the production of the data, and events and words 
that precede and follow the actual extracted pieces of data that are used to 
form the units for analysis. There is a very real danger for the researcher that 
in coding and categorizing the data the meaning of the data is lost or trans- 
formed by wrenching it from its location (1) within a sequence of data 
(e.g. interview talk) or (2) within surrounding circumstances which have a 
bearing on the meaning of the unit as it was originally conceived at the time 
of data collection. 

¢ There is the danger of oversimplifying the explanation. In the quest to identify 
themes in the data and to develop generalizations the researcher can feel 
pressured to underplay, or possibly disregard, data that ‘doesn’t fit’. 
Inconsistencies, ambiguities and alternative explanations can be frustrating 
in the way they inhibit a nice clear generalization — but they are an inherent 
feature of social life. Social phenomena are complex, and the analysis 
of qualitative data needs to acknowledge this and avoid attempts to 
oversimplify matters. 

e The analysis takes longer. The volume of data that a researcher collects will 
depend on the time and resources available for the research project. When it 
comes to the analysis of that data, however, it is almost guaranteed that it 
will seem like a daunting task. This is true for quantitative research as much 
as it is for qualitative research. But, as Bryman and Burgess (1994: 216) point 
out, when it comes to the analysis of seemingly vast amounts of quantita- 
tive data ‘the availability of standard statistical procedures and computer 
programs for handling them is generally perceived as rendering such data 
non-problematic’. The analysis of the data can be achieved relatively 
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quickly and the decision-making behind the analysis can be explained fairly 
succinctly. When researchers use qualitative data, however, the situation is 
rather different. Primarily, this is because qualitative data are generally 
unstructured when they are first collected in their ‘raw’ state (e.g. interviews, 
field notes, photographs). Computer programs can assist with the manage- 
ment of this data and they can even help with its analysis, but nowhere near 
to the extent that they can with quantitative data. In the case of qualitative 
data, the techniques used for the analysis are more time-consuming and the 
decisions made by the researcher are less easily described to the reader of 
the research. The result is that qualitative data takes considerably longer 
to analyse. 
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Checklist for the analysis of qualitative data 


When analysing qualitative data you should feel confident about answering 
‘yes’ to the following question: 


Does any report of the findings include an explicit 
account of the process of data analysis? 


Have sufficient details been incorporated to allow an 
‘audit’ of key decisions in the process of the research? 


Has the data analysis taken an iterative approach? 
Has the analysis been based on a depth scrutiny of the data? 


Has the data analysis included appropriate steps to 
verify the data? 


Have relevant details been supplied to permit readers to 
think about the transferability of the findings? 


Has the topic been approached with an ‘open mind’? 
Have alternative explanations been considered? 


Does the research report include some reflexive account 
of the researcher’s self and its possible impact? 


Is the choice of approach to data analysis suitable for the 
kind of qualitative data used (text, image)? 


Do the aims of the research match one or more of the following: 


Look for hidden messages — content analysis? 
Develop concepts or theories — grounded theory? 
Expose the workings of power through 

language — discourse analysis? 

Reveal underlying rules and structures of talk 

— conversation analysis? 

Accounts of the construction of self and social 
world — narrative analysis? 

Explain the meaning and cultural significance 
of images — image-based analysis? 
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WRITING UP THE 
RESEARCH 


Producing accounts of the research e General guidelines on style and 
presentation e The structure of research reports e The research 
methods chapter or section e Different audiences for research 


e The Harvard referencing system e Writing up qualitative research 
e Checklist for writing up the research 


‘Writing up’ is an integral part of the research process. It is not something 
tagged on at the end. Nor is it a simple, straightforward task. Far from it, 
writing up is skilful. It involves a blend of interpretation, craft and convention 
aimed at producing a formal record of the research that can be evaluated 
by others. Writing up, in essence, calls on researchers to exercise skill and 
judgement as they do the following: 


e produce an account of the research; 

e tailor reports to meet the requirements of different audiences; 

e adopt an appropriate style of writing and take account of certain technical 
conventions. 


Producing accounts of the research 


When researchers write up their work, they cannot produce a literal descrip- 
tion of what took place. It is almost impossible to envisage a literal account of 
the research process, because: 
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e There are always limitations to the space available to provide the account of 
what happened, which means the researcher needs to provide an edited 
version of the totality. Decisions need to be made about what details are 
included and which are considered less important and can be missed out of 
the account. 

e The editorial decisions taken by the researcher are likely to be shaped by 
the researcher’s need to present the methods in their best possible light. 
Quite rationally, the researcher will wish to put a positive spin on events 
and to bring out the best in the process. Without resorting to deceit 
or untruths, the account of research will almost certainly entail some 
upbeat positive filtering. The point, after all, is to justify the procedures as 
‘good’ research. 

e Although research notes will be used to anchor the description of what 
happened during the course of the research, the writing up is inevitably a 
retrospective vision. Situations and data are likely to have a different meaning 
when viewed from the end of the research process from that at the time they 
occurred. They will be interpreted with the wisdom of hindsight. 

e The impact of social norms and personal values on the way we interpret 
events pretty well guarantees that, to a greater or lesser extent, any account 
of research should be regarded as a version of the truth rather than a literal 
depiction of what happened. Within the social sciences, the idea of a purely 
objective position is controversial, and a researcher would be naive to pre- 
sume that his or her account can stand, without careful consideration, as an 
‘objective’ description of what really occurred. 


The end product, therefore, no matter how scrupulous it attempts to be, must 
always be recognized for what it is —- an account of the research. 


General guidelines on style and presentation 


The rules of style for writing up research are really like the rules for writing 
which operate for the English language in general. Inexperienced writers 
should stick to the rules. Experienced writers might break the rules, but the 
assumption is that they know they are breaking the rules and are doing 
so consciously for a particular purpose, to achieve a specific effect. Project 
researchers, then, are best advised to stick to the rules. 


Use the third person 


There are occasional circumstances where research might be written up using 
the first person: ‘I distributed 164 questionnaires ... I received 46 per cent 
back ... From the research I found that ...’. Sometimes this is to be found 
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in qualitative research when researchers are keen to emphasize their per- 
sonal involvement and the role of their personal identity in the collection 
and analysis of the data. Or, it might be used in the context of an infor- 
mal report intended for restricted distribution. It is more conventional, 
however, to write research reports in the third person: ‘Research involved the 
distribution of 164 questionnaires ... A response rate of 46 per cent was 
achieved . . . Findings from the research indicated that . . .’. This is particularly 
the case for formal reports such as dissertations and theses. 


Use the past tense 


For the most part, this convention poses little trouble because researchers are 
reflecting upon events that happened in the past. It might be the recent past, 
but none the less the writing up refers to things that were done and events that 
happened. 


Ensure good standards of spelling and grammar 


Perhaps obvious, this convention is still worth stressing. Word-processing 
packages can be used as spell-checkers and can help with the writing style. 
Scrutiny of the text by the researcher is still needed, however, to avoid those 
text errors which cannot be picked up by the spell-checker; where ‘at’ has been 
typed instead of ‘an’, for example. 


Develop logical links from one section to the next 


A good report is one that takes the reader on a journey of discovery. The 
pathways of this journey should be clear, and the reader should never be left in 
doubt about the direction of the discussion or the crucial points that are being 
argued. The logic of the discussion should build point on point towards a final 
conclusion. (See ‘The structure of research reports’, opposite.) 


Use headings and sub-headings to divide the text into clear sections 


Judicious use of headings and sub-headings can separate the text into blocks in 
a way that makes the reader’s task of understanding the overall report far 
easier. They act as signposts. As with signposts, too few and the reader gets lost, 
too many and the reader gets confused. As with signposts, their success 
depends on being clear and being in the right place. 


Be consistent in the use of the referencing style 


Whether the Harvard or the numerical style is used, there should be consistency 
throughout the report. Use one or the other, not both. Details of the Harvard 
style are presented on p. 320. 
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Use care with the page layout 


The visual element of the presentation is important, and the researcher should 
give some consideration to things like the page layout and the use of graphs, 
tables and illustrations to enhance the appeal of the report. 


Present tables and figures properly 


Tables and figures should be presented in a consistent style that provides the 
reader with the necessary information to decipher the meaning of the data 
contained in them. As indicated in Chapter 13, there should be: 


e aclear and precise title; 

e the source of the table or figure (if it is not original material); 
e the units of measurement being used (£, cm, tonnes, etc.); 

e x axis as the independent variable (where relevant). 


/\ Caution: breaking with convention 


If project researchers choose to break these conventions, they should 
be aware that they run the risk of having their report perceived as 
poorly written. If this is a risk that is deemed worth taking, they need to 
offer some explanation of why the rules have been broken to avoid 
any such impression. So, for example, if a researcher decides quite 
consciously to present an account of the research which uses the 
first person, there should be some acknowledgement that this is not 
conventional and some justification offered for its use. 


The structure of research reports 


Research reports tend to be structured according to certain conventions. 
The order in which they present material, and even the headings used, tend 
to conform to a familiar pattern - a pattern which is dictated largely by 
the need to present information in a logical order, with each new section 
building on information that has been provided earlier. The project researcher 
would do well to use such a structure for guidance when it comes to 
writing up. 

The familiar structure for research reports, in some contexts, has become 
formalized into a template for dividing up the material and presenting it in a 
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preordained sequence. Following the lead of scientific journals, there are jour- 
nals for social research which insist on the report conforming with the use of 
headings such as ‘Abstract’, ‘Introduction’, ‘Methods’, ‘Findings’, ‘Discussion’, 
‘Conclusions’. If researchers are writing for such journals, they must adopt this 
rigid format. Elsewhere, researchers can exercise a little more freedom in their 
construction of the research report, being more flexible with the order and 
using headings that are somewhat different. Writing up research for a PhD, for 
instance, allows some leeway from this structure, as does writing up a commis- 
sioned piece of research whose audience is likely to have different priorities. 
But, even where researchers do not find themselves constrained by explicit, 
externally imposed formats, there remains the same underlying rationale to 
the writing up of research, and this should guide the researcher. The con- 
ventional structure can and should be adapted to meet the requirements of 
specific audiences and specific kinds of research, but it equally provides a 
robust template that all social researchers can use to guide their construction 
of a research report. 

The conventional structure for reporting research divides the material into 
three parts: the preliminary part, the main text and the end matter. This is as 
true for a full length book as it is for a PhD, for a brief journal article and for a 
workplace project. 


The preliminary part 
Title 


The title itself needs to indicate accurately the contents of the work. It also 
needs to be fairly brief. A good way of combining the two is to have a two-part 
title: title and subtitle. The first part acts as the main title and gives a broad 
indication of the area of the work. The second part adds more detail. For 
example, ‘Ethnicity and friendship: the contrast between sociometric research 
and fieldwork observation in primary school classrooms’. 


Abstract 


An abstract is a synopsis of a piece of research. Its purpose is to provide a brief 
summary which can be circulated widely to allow other people to see, at a 
glance, if the research is relevant to their needs and worth tracking down to 
read in full. An abstract is normally about 250-300 words in length, and is 
presented on a separate sheet. 


Key words 


Researchers are often asked to identify up to five ‘key words’. These words are 
‘identifiers’ — words that capture the essence of what the report is all about. The 
key words are needed for cross-referencing during library searches. 


WRITING UP THE RESEARCH 313 


List of contents 


Depending on the context, this can range from being just a list of chapter 
headings and their starting page through to being an extensive list, including 
details of the contents within the major section of the report; for instance, 
based on headings and sub-headings. 


List of tables and figures 


This should list the titles of the various tables and figures and their locations. 


Preface 


This provides the opportunity for the researcher to give a personal statement 
about the origins of the research and the significance of the research for the 
researcher as a person. In view of the importance of the ‘self’ in the research 
process, the Preface offers a valuable place in the research report to explore, 
albeit briefly, how the research reflects the personal experiences and biography 
of the researcher. 


Acknowledgements 


Under this heading, credit can be given to those who have helped with the 
research. This can range from people who acted as ‘gatekeepers’ in relation 
to fieldwork, through to academic supervisors, through to those who have 
commented on early drafts of the research report. 


List of abbreviations 


If the nature of the report demands that many abbreviations are used in 
the text, these should be listed, usually alphabetically, under this heading, 
alongside the full version of what they stand for. 


The main text 


The main text is generally divided into sections. The sections might be 
chapters as in the case of a larger piece of work or headings as in the case 
of shorter reports. In either case, they are normally presented in the following 
order. 


Introduction 


For the purposes of writing up research there needs to be an introduction. This 
may, or may not, coincide with a section or chapter titled as an ‘Introduction’, 
depending on how much discretion is open to the researcher and how far this 
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is taken. The important thing is to recognize that, at the beginning, the reader 
needs to be provided with information about: 


e the background to the work (in relation to significant issues, problems, 
ideas); 

e the aims of the research; 

e key definitions and concepts to be used; 

e optionally, in longer pieces, an overview of the report (mapping out its 
contents). 


Literature review 


This may be presented as an integral part of the ‘Introduction’ or it may appear 
as a separate chapter or section. It is, though, essential that in the early stages 
of the report there is a review of the material that already exists on the 
topic in question. The current research should build on existing knowledge, 
not ‘reinvent the wheel’. The literature review should demonstrate how the 
research being reported relates to previous research and, if possible, how it 
gives rise to particular issues, problems and ideas that the current research 
addresses. 


Methods of investigation 


At this point, having analysed the existing state of knowledge on a topic, it is 
reasonable to describe the methods of investigation. See the section on ‘The 
research methods chapter or section’ (p. 318) for guidance on how this should 
be done. 


Findings 


This is where the reader is introduced to the data. Aspects of the findings are 
singled out and described. The first step is to say, ‘This is what was found 
with respect to this issue ... This is what was found with respect to another 
issue ...’. The aim for the researcher is to be able to present relevant findings 
before going ahead to analyse those findings and see what implications they 
might have for the issues, problems or ideas that prompted the research. First 
things first: let’s see what we have found. Then, and only then, as a subsequent 
stage, will we move on to considering what significance the data might have in 
the context of the overall aims of the research. 


Discussion and analysis 


Here, the findings that have been outlined are subjected to scrutiny in terms 
of what they might mean. They are literally discussed and analysed with 
reference to the theories and ideas, issues and problems that were noted earlier 
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in the report as providing the context in which the research was conceived. 
The researcher ‘makes sense’ of the findings by considering their implications 
beyond the confines of the current research. 


Conclusions and recommendations 


Finally, in the main text, the researcher needs to draw together the threads of 
the research to arrive at some general conclusion and, perhaps, to suggest 
some way forward. Rather than let the report fizzle out as it reaches the end, 
this part of the report should be constructive and positive. It can contain some 
of the following things: 


e aretrospective evaluation of the research and its contribution; 

e recommendations for improving the situation, guidelines or codes of 
practice; 

e identification of new directions for further research. 


The end matter 
Appendices 


This is the place for material which is too bulky for the main body of the text, 
or for material which, though directly relevant to the discussion, might entail 
too much of a sidetrack if placed in the text. Typical items that can be lodged 
in an appendix are: 


e extensive tables of data; 

e questionnaires used in a survey; 

e extracts from an interview transcript; 
e memos or minutes of meetings; 

e technical specifications. 


Notes 


These will mainly occur when the researcher is using a numerical referencing 
system. They also offer the opportunity for scholarly details to be added which 
would interrupt the flow of the reading were they to be put directly into the text. 


References 


See the section on the Harvard system of referencing on p. 320. 


Index 


Provision of an index is usually restricted to large reports and books. It is 
unlikely that the kind of report produced by a project researcher would require 
an index. 
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The research methods chapter or section 


In all accounts of research there needs to be some description and justifi- 
cation of the methods used to collect the data. In larger works, this appears 
in a separate chapter. In shorter reports and articles, it tends to be curtailed 
to a section under a ‘research methods’ heading or to a clearly identifiable 
paragraph or two. Within the confines of the available space, the researcher 
needs to explain how the research was conceived, designed and executed. 
This is vital in order for the reader to make some informed evaluation of 
the study. Basically, if the reader is not told how and why the data were 
collected, he or she cannot make any judgement about how good the 
research is and whether any credibility should be given to its findings or 
conclusions. 

Within the confines of the space available, the methods section should do 
three things. 


Describe how the research was conducted 


Precise details need to be given, using specific and accurate numbers and 
dates. 


e what method(s) were used (the technical name)? 

e when did the research take place (month and year, duration of research)? 

e where did the research take place (location, situation)? 

e how was access to the data or subjects obtained? 

e who was involved in the research (the population, sample, cases, examples)? 
e how many were involved in the research (precise numbers)? 

e how were they selected (sampling technique)? 


Justify these procedures 


An argument needs to be put forward supporting the choice of method(s) as: 


e reasonable in terms of the resources and time available; 

e appropriate under the circumstances for collecting the necessary type of 
data; 

e suitable for addressing the issues, problems or questions that underpin the 
research; 

e having rigour, coherence and consistency — a professional standard; 

e producing data that are valid; 

e using methods that are reliable; 

e conforming with ethical standards. 
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Acknowledge any limitations to the methods employed 


Good research evaluates the weaknesses as well as the strengths of its meth- 
odology. When writing up, the researcher should acknowledge any: 


e inherent limitations of the methodology; 

e scope of what can, and cannot, be concluded on the basis of the research 
that was undertaken; 

e ways in which resource constraints had a direct influence on the volume or 
kind of findings produced; 

e reservations about the authenticity, accuracy or honesty of answers; 

e ways in which, in retrospect, alternative methods might have proved to be 
more useful; 

e unexpected factors which arose during research that influenced the 
outcome. 


Different audiences for research 


Different audiences have different expectations when it comes to reading the 
report, and the researcher needs to decide how to pitch the account of the 
research to meet the expectations of the group whose views are considered 
most important. The easy illustration of this comes in the form of research 
reports produced for an academic qualification as part of an examined course 
— an undergraduate project, a master’s dissertation, a doctoral thesis. The 
researcher should need little reminding that the work will be assessed by 
supervisors and examiners who will be focusing on detail, rigour, precision, 
coherence and originality as top priorities. A different audience might bring 
different expectations. In the case of commissioned research, the audience is 
likely to be more concerned with receiving a report which is succinct, easy to 
digest and strong on practical outcomes. In principle, the research could be 
just the same; the way it is written up, though, will reflect the needs of the 
differing audiences. This will affect: 


e the style of presentation; 

e the detail and length of the account; 

e the amount of technical detail included; 
e the terminology used. 


Good practice: meeting the readers’ expectations 


Research reports should be tailored to meet the expectations and abilities of 
the audience for whom they are written. 
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Requirements in terms of style of referencing and some other technical aspects 
associated with research reports can vary in their detail. There are some general 
sources of information on this. For the production of dissertations there is, for 
example, the British Standards specification no. 4821. Although this was 
withdrawn in 1990, it has not been superseded and is still recommended by 
the British Library. For the production of academic articles and for referencing 
techniques, the researcher could turn to the Publication Manual of the American 
Psychological Association. There are also books devoted to guidance for authors 
on the technical conventions associated with writing up research — for exam- 
ple, K. L. Turabian’s Manual for Writers of Term Papers, Theses and Dissertations 
(Chicago: University of Chicago Press, 7th edn, 2007). However, each pub- 
lisher and each university will have its own specific requirements. Publishers 
always include guidance to authors which spell out their own particular policy 
as far as the presentation of work is concerned. University regulations, like- 
wise, will contain details covering the style and presentation of dissertations 
and theses. The crucial thing for the project researcher to bear in mind is that 
their research report should always adhere exactly to the formal requirements 
of the principal ‘audience’. 


Good practice: identifying the appropriate report style 


Researchers should identify which specific technical style they are expected to 
adopt for the purposes of writing their report, and follow it meticulously. 


The Harvard referencing system 


There are conventions for referring to the ideas, arguments and supporting 
evidence gleaned from others. There are two that are generally recognized: 
the Vancouver system, i.e. numerical system, and the Harvard system. The 
numerical system involves placing a number in the text at each point where 
the author wishes to refer to a specific source. The full references are then given 
at the end of the book or individual chapters, and these can be incorporated 
into endnotes. It is the other system, the Harvard system, however, which is 
more common in social research these days and, for that reason, further details 
will concentrate on this convention. 

In the Harvard system, the sources of ideas, arguments and supporting 
evidence are indicated by citing the name of the author and the date of pub- 
lication of the relevant work. This is done at the appropriate point in the text. 
Full details of the author’s name and the publication are subsequently given at 
the end of the report, so that the reader can identify the exact source and, if 


WRITING UP THE RESEARCH 319 


necessary, refer to it directly. As used in this book, the Harvard system involves 
referring to authors in the text in the following ways: 


e Baker (2010) argues that postmodernism has a dubious future. 

e It has been argued that postmodernism has a dubious future (Baker 2010). 

e The point has been made that ‘it is not easy to see what contribution 
postmodernism will make in the twenty-first century’ (Baker 2010: 131). 


In the References section towards the end of the research report, the full details 
of ‘Baker 2010’ are given, as they are for all the authors’ works cited in the 
report. For Baker, it might look like this: 


Baker, G. (2010) The meaning of postmodernism for research methodology, 
British Journal of Health Research, 25: 249-66. 


As far as the References section is concerned, there are seven key com- 
ponents of the Harvard system: 


e Author’s name and initial(s). Alphabetical order on authors’ surnames. 
Surname followed by forename or initial. If the book is an edited volume, 
then (ed.) or (eds) should follow the name. 

e Date of publication. To identify when the work was written and to dis- 
tinguish different works published by the same author(s). 

e Title. The title of a book is put in italics, and uses capital letters for the first 
letter of the main words. Papers and article titles are not in italics and have 
titles in lower case. 

e Journal name (if applicable). This is put in italics and details are given of the 
number, volume and page numbers of the specified article. If the source is a 
contribution to an edited volume, then details are given about the book in 
which it appears (i.e. editor’s name, title of edited volume). 

e Publisher. Vital for locating more obscure sources. This is included for books 
but not for journals. 

¢ Place of publication. Helpful in the location of obscure sources. 

e Edition. If the work appears in a second or subsequent edition this needs to 
be specified. 


Real examples are to be found in the References section in this book, and can 
be used to illustrate the principles further. 


Good practice: use of referencing software 


Researchers should make use of software packages to help with the organiza- 
tion of their references and with citing sources they refer to in their reports. 
Packages like EndNote allow the researcher to adhere to any one of the many 
styles that might be required. 
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Writing up qualitative research 


For the qualitative researcher, the presentation of research in sections under 
the headings of ‘abstract’, ‘introduction’, ‘findings’, ‘methods’, ‘discussion’ 
and ‘conclusions’ might seem inappropriate and not in keeping with the way 
the research actually evolved. Headings like these might seem to accord with a 
design and execution of research more in line with experiments and surveys 
than ethnography or grounded theory, and therefore pose a difficulty for the 
qualitative researcher when it comes to meeting the conventions associated 
with writing up research. They would seem to be artificial and inappropriate. 

Rather than ditch such headings, however, qualitative researchers might 
well consider using them as a template for constructing their accounts of the 
research — a template which gives some structure to the accounts and which is 
comfortably recognized by those coming from different traditions within the 
social sciences. While acknowledging that writing up qualitative research 
involves much more of a retrospective reconstruction of what actually hap- 
pened than would be the case with more positivist approaches, it still needs 
to be recognized as just that - a retrospective account rather than a literal 
depiction of the rationale and the events. By latching on to the traditional 
conventions, the interpretive social researcher is provided with a template for 
reporting the research. The template, in this case, does not provide a means 
for faithfully reporting in some structured sequential manner what actually 
happened in the process of research. It does, however, provide a means for 
reconstructing and presenting the research in a way that: 


e addresses and highlights the key issues; 
e is clearly comprehensible to the reader; 
e is logically ordered. 
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Checklist for writing up the research 


When writing up the research you should feel confident about answering 
‘yes’ to the following questions: 


Is there a suitable structure and logical development to 
the report? 


Is the text written in a clear style, free of spelling and 
grammatical errors? 


Does the writing style meet the expectations of the main 
audience for the research? 


Have the necessary conventions been followed in the 
writing up of the research? 


Are the references complete and do they follow a 
recognized style (e.g. Harvard)? 


Are the tables, figures, illustrations and diagrams 
properly labelled? 


Has a detailed and precise description of the research 
process been provided? 


Has the choice of method(s) been justified in relation to 
the type of data required and the practical circumstances 
surrounding the research? 


Have the limitations of the research methodology been 
acknowledged? 


© M. Denscombe, The Good Research Guide. Open University Press. 


Frequently asked questions 


What is analysis? 


Analysis means the separation of something into its component parts. To do 
this, of course, the researcher first needs to identify what those parts might be, 
and this links with a further meaning of analysis, which is to trace things back 
to their underlying sources. Analysis, then, involves probing beneath the sur- 
face appearance of something to discover the component elements which 
have come together to produce it. By tracing things back in this fashion, the 
researcher aims to expose some general principles that can be used to explain 
the nature of the thing being studied and can be applied elsewhere to other 
situations. See pp. 237-242. 


What is a concept? 


A concept is a basic idea. It is an idea that is generally abstract and universal 
rather than concrete and specific. And it is basic in the sense that it cannot be 
easily explained in terms of other ideas or equated to other ideas. In terms of 
ideas, then, a concept is a basic building block that captures the essence of a 
thing (e.g. ‘love’, ‘relevance’). See pp. 115, 116, 124, 286-288. 


Can you generalize from a case study? 


The answer is ‘yes’ — but it needs to be done cautiously. If the case study is 
properly conducted, it is possible to consider the implications of the findings 
from the specific case(s) for other examples of the kind of thing that is 
being studied. The case study researcher needs to acknowledge the limits to 
how far findings can be generalized and to recognize that this is not the same 
thing as the ‘generalizability’ of survey findings. See pp. 60-62 for a fuller 
discussion. 
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What are dependent and independent variables? 


An independent variable is the one that has the impact on the dependent 
variable. A dependent variable alters as a result of changes to the independent 
variable. It literally ‘depends’ on the independent variable. Any change in the 
dependent variable does not affect the independent variable. See pp. 66-67, 
257-261 for further details. 


What is a focus group? 


Focus groups make particular use of group dynamics. They consist of small 
groups of people who are brought together by a ‘moderator’ (the researcher) to 
explore attitudes and perceptions, feelings and ideas about a specific topic. Note 
that focus groups are not the same thing as a group interview. See Appendix 4. 


When should | use internet research methods? 


The Internet offers some fabulous possibilities for research. It holds the pro- 
spect of gathering huge volumes of data, of operating at a global level, of 
getting information quickly - and doing all this at relatively low cost. These 
obvious attractions, however, should not seduce the researcher into using 
the Internet on every possible occasion, irrespective of whether it happens 
to be the best tool for the job. There are pros and cons to using the Internet, 
just as there are with other research strategies (see Hine 2005). In particular, 
researchers need to consider the benefits for data collection that the new 
technology offers when weighed against methodological concerns related to 
things like: 


e sample bias; 

e interaction effect; 
e response rate; 

e ethics. 


A decision on whether it is appropriate to use ‘e-research’ should be based on 
an awareness of the implications of these factors and upon an evaluation of 
the respective advantages and disadvantages in relation to the specific topic 
that is to be investigated. See pp. 170, 190-192 (interviews), pp. 220, 224, 225, 
235 (websites), pp. 81, 82 (ethnography), p. 159 (questionnaires), and pp. 13, 
14, 21, 26, 159 (surveys). 
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What is the observer effect? 


When people are aware that they are being observed, there is a likelihood 
that they might alter their behaviour. One reason for this is that being 
observed can make people feel self-conscious and possibly embarrassed. 
Another reason is that they might feel threatened by being observed and act in 
a defensive manner. Either reaction threatens the validity of the observations 
because people are acting ‘unnaturally’. This observer effect means much the 
same as the ‘interviewer effect’ and the ‘halo effect’. See pp. 63, 68, 69, 142- 
143, 178-180, 204, 229 for further details. 


What is positivism? 


Positivism is an approach to social research which seeks to apply the natural 
science model of research to investigations of the social world. It is based 
on the assumption that there are patterns and regularities, causes and con- 
sequences in the social world, just as there are in the natural world. These 
patterns and regularities in the social world are seen as having their own 
existence — they are real. For positivists, the aim of social research is to discover 
the patterns and regularities of the social world by using the kind of scientific 
methods used to such good effect in the natural sciences. See pp. 94, 122, 148. 


What is pragmatism? 


As a philosophy for research, pragmatism is an approach that takes the 
research problem as its fundamental concern and it assesses the value of 
research strategies and methods in terms of how well they work at dealing with 
that problem. It is associated with the mixed methods approach (see Chapter 8) 
and stresses that it is not a matter of whether quantitative research is better 
than qualitative research or vice versa; it is a matter of which is more effective 
in relation to the research problem. Pragmatism in the context of research is 
not to be confused with any use of the term in everyday language which carries 
with it the idea of expediency or short-cuts. See pp. 4-6, 45-47, 109, 121. 
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What is quantitative research? 


The aim of ‘quantitative research’ is to measure phenomena so that they can 
be transformed into numbers. Once the phenomena have been quantified, 
they lend themselves to quantitative analysis or to analysis using statistical 
procedures. The goal of quantitative approaches, then, is to generate data that 
are numerical — with transforming what is observed, reported or recorded into 
quantifiable units. See Chapter 13. 


What is qualitative research? 


Qualitative research relies on transforming information from observations, 
reports and recordings into data in the form of written words or visual images, 
not numbers. The sources of information, it should be noted, need not 
differ between quantitative and qualitative approaches. A questionnaire, for 
example, can be used to produce either quantitative data (numbers) or qualita- 
tive data (words). See Chapter 14. 


What is reflexivity? 


Reflexivity concerns the relationship between the researcher and the social 
world. Contrary to positivism, reflexivity suggests that there is no prospect of 
the social researcher achieving an entirely objective position from which to 
study the social world. This is because the concepts the researcher uses to make 
sense of the world are also a part of that social world. Reflexivity is an awkward 
thing for social research. It means that what we know about the social world 
can never be entirely objective. A researcher can never stand outside the social 
world they are studying in order to gain some vantage point from which to 
view things from a perspective which is not contaminated by contact with that 
social world. Inevitably, the sense we make of the social world and the mean- 
ing we give to events and situations are shaped by our experience as social 
beings and the legacy of the values, norms and concepts we have assimilated 
during our lifetime. And these will differ from person to person, culture to 
culture. See pp. 86, 133, 327. 
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What is reliability? 


Researchers need to feel confident that their measurements are not affected by 
a research instrument that gives one reading on the first occasion it is used and 
a different reading on the next occasion when there has been no real change in 
the item being measured. This is why they are concerned with the ‘reliability’ 
of a research instrument. A good level of reliability means that the research 
instrument produces the same data time after time on each occasion that it is 
used, and that any variation in the results obtained through using the instru- 
ment is due entirely to variations in the thing being measured. None of the 
variation is due to fluctuations caused by the volatile nature of the research 
instrument itself. So a research instrument such as a particular experiment or 
questionnaire is said to be ‘reliable’ if it is consistent, and this is generally 
deemed to be a good thing as far as research is concerned. 


What is a research paradigm? 


The notion of a research paradigm refers to a set of beliefs and practices associ- 
ated with a particular style of research. It captures the idea that research is 
generally conducted in accord with a specific philosophy and world-view and 
that those who operate within the research paradigm share those views. 
Researchers working within each paradigm will share a general view about 
social reality (ontology) and the best tools for social research (epistemology). 
The paradigms can be very broad, or they can be quite restricted in scope. The 
most obvious examples of broad research paradigms are ‘quantitative research’ 
and ‘qualitative research’. 


What is the difference between research strategies and 
research methods? 


Strategies are broad approaches to research that are concerned with the direc- 
tion and scale of a research project and its underlying philosophy. Strategies 
involve the aims and design principles that shape the overall investigation. 
Research methods are tools used for the collection of empirical data. They 
are the means by which the researcher collects or produces different kinds 
of data. See pp. 3, 4. 
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What sample size do | need? 


If you are using a random sample as part of a large survey, then there is stat- 
istical calculation for the precise number needed for the survey. The size of the 
sample will depend on the level of accuracy that is required, and there are 
plenty of websites which provide a facility that will do the maths. Try a web 
search on ‘calculate sample size’. For qualitative research and small-scale 
research, however, the exact sample size is more difficult to estimate. It will 
reflect the time and resources available, and the number of suitable people 
who can be identified and contacted for inclusion. As a rule of thumb the 
sample should be sufficient in size for the purposes of the research and be 
comparable with the sample size of similar pieces of research. See pp. 40-48 for 
further information on this point. 


What is a theory? 


A theory is a proposition about the relationship between things. In principle, a 
theory is universal, applying at all times and to all instances of the thing(s) in 
question. In the social sciences, the notion of ‘theory’ needs to be treated more 
cautiously than in the natural sciences, because of the complexity of social 
phenomena and because people react to knowledge about themselves in a way 
that chemicals and forces do not. 


What is theoretical sampling? 


Used within a grounded theory approach, this is ‘[The] process of data collec- 
tion for generating theory whereby the analyst jointly collects, codes, and 
analyses his data and decides what data to collect next and where to find them, 
in order to develop his theory as it emerges’ (Glaser and Strauss 1967: 45). 
See pp. 36, 112-114, 124. 


What is theoretical saturation? 


In the grounded theory approach theoretical saturation signals the end of data 
collection and analysis relating to a specific concept or category. It occurs 
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when there is no advantage to further data collection because ‘additional 
analysis no longer contributes to discovering anything new about a category’ 
(Strauss 1987: 21). See pp. 117, 124. 


What is a thick description? 


A ‘thick description’ (Geertz 1973) aims to describe particular acts or events in 
relation to their cultural context. The emphasis is not just on describing what 
‘is’ but on explaining how the nature of this phenomenon is closely linked to 
other aspects of its social context. To illustrate what this means, Geertz himself 
draws on the example of ‘winking’. Simply describing the action itself is ‘thin 
description’. It is important to include as part of the description, but if we want 
to understand the meaning of the action we need a ‘thick description’ which 
supplies information about the intentions of the person doing the winking 
and the symbolic significance of the gesture in different settings. Only then 
can we understand whether the action is to be construed as a twitch, as a 
friendly gesture, as flirting, as a subtle conspiracy or whatever. 


What is validity? 


In a broad sense, validity means that the data and the methods are ‘right’. In 
terms of research data, the notion of validity hinges around whether or not the 
data reflect the truth, reflect reality and cover the crucial matters. In terms of 
the methods used to obtain data, validity addresses the question, ‘Are we 
measuring suitable indicators of the concept and are we getting accurate 
results?’ The idea of validity hinges around the extent to which research data 
and the methods for obtaining the data are deemed accurate, honest and 
on target. See pp. 299-306. 


Appendix 1 


RESEARCH ETHICS 


A practical guide 


Social researchers are expected to approach their task in an ethical manner. 
On moral grounds, this expectation stems from the belief that the public 
should be protected from researchers who might be tempted to use any 
means available to advance the state of knowledge on a given topic. It rests 
on the assumption that researchers have no privileged position in society 
that justifies them pursuing their interests at the expense of those they are 
studying - no matter how valuable they hope the findings might be. 


The need for Ethics Committee approval 


The importance attached to research ethics is evident in the fact that social 
researchers will normally need to get prior approval for their investigation 
from an Ethics Committee. In particular, this is the case where the proposed 
research involves collecting data directly from people, or collecting personal 
data about living people. Even with small-scale research, it is increasingly the 
case that before the research can be conducted it will need to have been scru- 
tinized by a committee of suitably qualified experts to ensure that the design 
of the research includes appropriate measures to protect the interests of the 
people and groups covered by the research. Researchers in the field of health 
will find that medical ethics committees are powerful in this respect, and those 
conducting investigations as part of any academic qualification will normally 
be required to have their plans vetted by an ethics committee within their 
university before they are allowed to start the investigation. Submissions 
to ethics committees will be judged according to the broad principles for eth- 
ical research outlined below, and might include further specific stipulations 
depending on the field of the research. And the need to get approval from such 
ethics committees reinforces the point that a concern with ethics is not an 
option — it is a fundamental feature of all good research. 
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Good practice: allow time for ethics approval 


Approval from an ethics committee should be obtained before any primary 
data is collected from the people involved in the research. The approval pro- 
cess can take weeks, sometimes months, to complete and this needs to be 
taken into consideration when planning the data collection phase of the 
research. 


Codes of research ethics 


Professional research associations almost always publish a code of conduct 
that they expect their members to abide by. Research funding bodies and 
government departments that commission research have similar guidelines 
governing acceptable practice in the context of research. There is some differ- 
ence in emphasis between these codes. Market research associations approach 
things in a slightly different way from medical associations, reflecting the 
different nature of the procedures the researchers are likely to use and the 
potential harm that might be done to participants. Market researchers collect- 
ing data on consumer preferences are not likely to be exposing participants 
to the same kind of risk, psychological or physical, as health researchers 
investigating the effectiveness of a new drug or surgical procedure. 


Good practice: a specific code of research ethics should be identified 
and used 


Project researchers should identify a code of ethics that represents their aca- 
demic discipline or professional association. This code should be used in 
relation to the research and, importantly, should be explicitly referred to in any 
account of the methods used in the research. 


The key principles of research ethics 


There are three key principles which underlie codes of research ethics. These 
principles have their origins in medical research and were established initially 
in response to atrocities committed during the Second World War which pur- 
ported to have been done in the name of science and the progress of medical 
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knowledge. Through the Nuremberg Code (1947-49), and later the Declaration 
of Helsinki (1964), the international community established the foundations 
for research ethics and enshrined the fundamental principle that the ends do 
not justify the means in the pursuit of knowledge. 

The Nuremberg Code and the Declaration of Helsinki have been developed 
over the years and various professional bodies, academic disciplines and 
government departments have adapted them for use in the context of 
social research as distinct from medical research. These codes of ethics for 
social research stick to the core principle (that in their pursuit of knowledge 
researchers must not ruthlessly use any means at their disposal) and they 
develop this along four lines. Social researchers are expected to conduct their 
investigations in a way that: 


e protects the interests of the participants; 

e ensures that participation is voluntary and based on informed consent; 
e avoids deception and operates with scientific integrity; 

e complies with the laws of the land. 


Principle 1: Participants’ interests should be protected 


There is general agreement that people should not suffer as a consequence of 
their involvement with a piece of research. Those who contribute to research 
as informants or as research subjects should be no worse off at the end of their 
participation than they were when they started. Nor should there be longer- 
term repercussions stemming from their involvement that in any sense harm 
the participants. 

In accord with this principle, researchers have a duty to consider in advance 
the likely consequences of participation and to take measures that safeguard 
the interests of those who help with the investigation. First of all, they need to 
ensure that participants do not come to any physical harm as a result of the 
research. With some kinds of research there needs to be careful consideration 
of the physical safety of participants, anticipating any threats arising from the 
time/location of meetings, travel to such locations, or even the possibility of 
the participant suffering retribution from people who feel threatened by 
the participant’s involvement in the research or the disclosure of particular 
information. The possibility might seem quite remote, but researchers have 
a responsibility to anticipate such risks and avoid any physical harm from 
occurring. 

Second, care needs to be taken to avoid psychological harm resulting from 
research. Again, this consideration should be built into the design of the 
research because any investigation that is potentially going to lead to trauma, 
stress or other psychological harm will not be considered ethical. On this 
point, researchers need to bear in mind the extent to which the investigation 
will be intrusive, touch on sensitive issues or threaten the beliefs of the 
participants. 
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Third, participants should suffer no personal harm arising from the dis- 
closure of information collected during the research. Disclosure of personal 
information could be embarrassing for the participants and, depending on the 
type of information, it could also involve economic loss or even the prospect 
of legal action being taken against participants on the basis of the disclosed 
information if it was to come to the attention of relevant authorities (e.g. 
employers, Inland Revenue, social security inspectors, the police, customs offi- 
cials). There is broad agreement, therefore, that researchers need to protect the 
interests of the participants by ensuring the confidentiality of information 
that is given to them. When publishing results care needs to be taken not 
to disclose the personal identities of individuals who have contributed to 
the findings. 


Good practice: protection of the participants’ interests 
Researchers should: 


e anticipate any threats to the personal safety of participants, and take steps 
to avoid these occurring; 

e respect participants’ privacy and sensitivities, and avoid undue intrusion 
when collecting data; 

e treat all information disclosed to them during research as confidential, and 
not disclose it to other participants or people not connected with the 
research; 

e guarantee the anonymity of individuals and organizations in any published 
documents arising from the research — unless participants give explicit (and 
written) consent to identify them by name. 


Principle 2: Participation should be voluntary and based on informed consent 


People should never be forced or coerced into helping with research. Their 
participation must always be voluntary, and they must have sufficient infor- 
mation about the research to arrive at a reasoned judgement about whether or 
not they want to participate. These are the premises of ‘informed consent’. 
Particularly where the kind of information collected is of a sensitive nature, 
there is a need to get informed consent. But, more than this, there is often a 
need to get that consent in writing. When the consent is in writing it acts as a 
way of formally recording the agreement to participate and confirming that 
the participant has been informed about the nature of the research. This pro- 
tects the researcher from any possible accusation that he or she acted 
improperly when recruiting people to take part in the research. 
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Good practice: informed consent 
For participants to be able to give informed consent researchers need to: 
e make it very clear that participation is voluntary; 


e provide participants with adequate information about the research; 
e specify what kind of commitment is being required of participants. 


A formal way to confirm that participation is on the basis of informed consent 
is to use a consent form (see the template below). 


The consent form — a template 


Identity of the researcher 


Participants are entitled to know who is conducting the research and where 
they can be contacted. The consent form, therefore, should include 


e the name(s) of the researcher; 

e an address and other contact details at which he or she can be contacted; 

e where appropriate, the name of the organization under whose auspices the 
research is being conducted. 


Information about the research 


There needs to be a statement providing enough detail for the participant to 
understand, in broad terms, the aims, methods and anticipated outcomes of 
the research. This must be brief since participants will not normally wish to 
spend a lot of time reading page after page of details. Some researchers pres- 
ent the information on a separate sheet that can be left with the participant for 
future reference. At other times, the statement is limited to a few sentences on 
the consent form itself. Either way, participants need to be provided with a 
brief description of: 


e what the research is investigating; 
e how it is to be conducted; 
e what benefits are likely to emerge from the investigation. 


Expectations about the participant’s contribution 


Informed consent presumes that participants know in advance what tasks the 
researcher expects them to undertake and what, if any, reward they might 
expect for their efforts. They will need to be told whether they are expected to 
take part in an interview, take part in a laboratory experiment or complete a 
questionnaire. And they should be given some indication of how much time 
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this is likely to take. They might also need to know whether any expenses they 
incur will be paid for by the researcher, and whether they will receive some 
kind of payment or other incentive (such as a free copy of the final report of the 
research). The consent form, then, should specify: 


e what tasks the participant will undertake; 
e how much of their time this is likely to take; 
e what rewards or incentives there might be for taking part in the research. 


The right to withdraw consent 


Signing a consent form does not oblige participants to continue in the role if, 
at some subsequent stage, they no longer wish to do so. The consent form is 
not a contract that binds a person to the task of helping with the research — 
and this should be made clear on the consent form itself. Participants should 
be made aware that, should they wish to do so, it is their right to withdraw their 
consent at any time. 


Confidentiality and security of data 


The consent form needs to incorporate some commitment by the researcher 
with regard to the conduct of the research and the treatment of the data. Any 
agreement to take part in research ought to be made with an understanding of 
what code of ethics the researcher will use and what measures will be taken 
to safeguard the data and protect participants from any disclosure of their 
individual identity. Such assurances could include: 


e specifying which professional association’s code of ethics will be used to 
during the course of the research; 

e what security measures will be taken to ensure the confidentiality of 
the data; 

e how anonymity of the data will be guaranteed (if appropriate). 


Signature of the participant, with date 


This signature can incorporate an acknowledgement that the participant has 
been advised about the requirements of the research and understood what is 
involved (e.g. ‘I have read and understood the written details provided for me 
about the research, and agree to participate in the project. Signed ...’). 


Counter signature of the researcher, with date 

It is good practice for the researcher, as well as the participant to sign the 
consent form effectively committing the researcher to uphold his or her side of 
the agreement. 


APPENDIX 1 RESEARCH ETHICS 335 


What if it is not possible to get written consent? 


It may not always be possible to gain consent from all those who take part in a 
research project. Research on crowd behaviour at football matches or observa- 
tion of the movements of visitors to art galleries are clearly instances where it 
would prove impossible to get everyone to complete a consent form. There are 
other occasions where it may not be deemed practical or, indeed, necessary to 
get written consent. Approaching people in the street to ask a few questions 
about their shopping habits, for example, might not be seen as requiring the 
completion of a consent form. In this case there is some tacit consent. The 
agreement to answer the questions clearly indicates that their participation is 
voluntary and, because the nature of the participants’ involvement in the 
research is rather ‘low risk’, under the circumstances this ‘verbal consent’ 
might be deemed sufficient. The codes of conduct of the professional associ- 
ations generally recognize this and include clauses that acknowledge that 
exceptions might need to be made to the general rule about written consent. 
Normally, the conditions under which researchers can collect data ethically 
without getting written consent are those where: 


e itis not realistically possible to get such consent; and 
e the form of data collection is unlikely to involve much by way of personal 
risks to the informant. 


Principle 3: Researchers should operate in an open and honest manner with 
respect to the investigation 


Researchers are expected to avoid deception or misrepresentation in their deal- 
ings with participants. They are expected to be open and explicit about what 
they are doing - to let people know that they are researchers and that they 
intend to collect data for the purposes of an investigation into a particular 
topic. Furthermore, they are expected to tell the truth about the nature of their 
investigation and the role of the participants in that research. 


Good practice: openness in the conduct of research 
Researchers should: 


e produce a brief summary of the aims of the research and the nature of the 
data that are to be collected. This should be no more than one page in 
length and written without jargon so that the purpose of the research can be 
clearly understood by potential participants and others who might have an 
interest in the topic of the research; 

e where possible, develop a website for the project so that information 
about the project can be widely available to interested parties. Links from 
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the Home page can be used to provide further information about the 
researchers (their credentials and experience) and more details about the 
background to the project for those who might want to know more. 


There is also an expectation that researchers operate with ‘scientific integ- 
rity’. This means, in effect, that researchers are expected to uphold the highest 
standards of professionalism and honesty in terms of their treatment of the 
data and dealings with other researchers. On the first point, working ethically 
means doing the best possible job to collect and analyse data in an objective 
and honest way. Researchers should not be swayed to supply findings that suit 
the needs of those who fund or sponsor the research. Nor should they allow 
personal preferences and ideals to cloud their judgement and get in the way of 
providing a balanced and dispassionate interpretation of their findings. They 
should be ‘fair’ and unbiased. On the second point, there is an expectation 
that researchers should acknowledge the contribution of others and not 
‘steal’ their work through plagiarism. Researchers certainly need to ‘use’ the 
findings of others - either by way of building upon it or by criticizing it — but 
when doing so they should always explicitly acknowledge the source of the 
other work. 

Scientific integrity covers two further obligations on the researcher. First, the 
researcher is expected to have the appropriate experience and competence to 
undertake the investigation. Without this there is a danger that the partici- 
pants might be harmed. Thinking of medical research the point behind this is 
easy to see. With social research too, however, Ethics Committees will look to 
be reassured that researchers will restrict their investigations to those styles of 
research for which they are qualified and for which their experience makes 
them well prepared. Linked with this point, there is a clear expectation that 
researchers should not engage in research which is frivolous or whimsical, or 
for which there is not a good ‘scientific’ reason to believe that something 
of value will be learnt as an outcome. Behind both these points there is the 
principle that participants should not be exposed to the risk of harm or 
inconvenience by being asked to participate in research that is not likely to be 
beneficial. This is the principle of ‘beneficence’. 


Good practice: honesty and integrity in the conduct of research 
Researchers should: 


e avoid deception or misrepresentation in their dealings with participants; 

e provide fair and unbiased interpretation of their findings; 

e explicitly acknowledge any sponsorship or vested interests relating to the 
research; 
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e avoid plagiarizing other researchers’ work; 
e have the necessary skills and experience to carry out the research; 
e show that real benefits are likely to emerge from the research. 


What if some deception is necessary? 


There are occasions when it is not feasible for the researcher to be completely 
open and honest with participants. To be so would automatically jeopardize 
any prospect of the investigation actually taking place or would cause partici- 
pants to react to the research in a way that would contaminate the results. 
Many psychology experiments, for example, would be ruined if the subjects 
knew in advance what the researcher was investigating, and it has been esti- 
mated that about three-quarters of psychology experiments involve giving 
participants information that, in one respect or another, actually misleads 
them about the purpose of the experiment. Some types of ethnographic study, 
likewise, involve fieldwork where the researcher needs to disguise his/her 
true role because if people knew that research was being conducted they would 
act differently from normal, and the ‘naturalness’ of the setting would be 
destroyed. 

When some form of deception is seen as necessary, researchers face an 
ethical dilemma: from the point of view of good research design deception 
is regarded as essential, yet from the point of view of ethical research such 
deception is bad practice. The codes of conduct published by professional 
associations, however, generally acknowledge that, on certain occasions, some 
deception by researchers might be warranted as absolutely necessary for the 
purposes of the research. Such occasions, though, need to be treated as excep- 
tions to the rule - ones that can be justified only on the basis of the very special 
circumstances surrounding the particular investigation. The conditions under 
which it might be acceptable generally require the researcher: 


e to provide an explicit justification for why the deception is vital and why 
there is no suitable alternative approach that would permit full disclosure 
about the nature of the research; 

e where appropriate, to use a ‘debriefing’ session in which, after the event, 
participants are put in the picture about the true nature of the enquiry and 
why they could not be informed about this at the beginning. 


Principle 4: Research should comply with the laws of the land 


Researchers do not stand above the law nor should they put themselves out- 
side the law. Operating within the law, of course, does not necessarily make an 
action ethical. Indeed, there are many actions a researcher could take that 
might be perfectly legal but which would be considered to be absolutely 
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unethical in the context of conducting research. And a case could be made that 
a researcher who breaks the law might, nonetheless, be doing something that 
is ethical. Like an investigative journalist who needs to use unscrupulous 
methods in order to expose a case of corruption or organized crime it is pos- 
sible that a social researcher might make a similar case for stepping outside the 
law. However, this would be a dangerous thing for any but the most experi- 
enced researchers and for project researchers would be most ill-advised. Most 
codes of research ethics are quite clear on the point: social researchers are 
expected to abide by the laws of the land. 

Bearing this in mind, there are some laws which of particular relevance for 
social researchers. First, there are those laws that set bounds to what is accept- 
able in terms of the topic of the research. There are areas where the nature of 
any research project is likely to ‘cross the line’ in terms of the law and, for 
this reason project researchers should steer well clear of such areas. Obvious 
examples would be the way that research involving empirical investigation 
into topics such as terrorism, drug trafficking, paedophilia, prostitution or 
pornography could all too easily, and unintentionally, end up breaking the 
law. Second, there are certain ways of collecting data that could fall foul of the 
law. There are data protection laws that need to be considered, laws against 
hacking into computers, laws against fraud, and copyright laws all of which 
set boundaries around what can be legitimately used as a means for collecting 
data. The data protection laws also have a bearing on the way that researchers 
need to store their data and the extent to which personal data can be distrib- 
uted to third parties not actually involved with the research. (See Appendix 2 
Data Protection: a practical guide.) 


Good practice: compliance with the law 
To help them stay within the law, researchers should: 


e avoid researching sensitive topics where the methods of data collection and 
analysis are likely to involve breaking the law; 

e take care on matters relating to the ownership of the data, intellectual 
property rights and copyright matters; 

e take reasonable measures to ensure that data are kept private and secure. 


Ethics and Internet research 


Internet-based research involves the same principles of research ethics as all 
other research but it brings with it some issues that warrant particular atten- 
tion. A considerable amount of attention has been paid to the new kinds of 
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challenge faced by Internet research in terms of things like informed consent, 
privacy and deception, confidentiality and legislation (e.g. Langford 2000; Ess 
2002; Buchanan 2004; Markham and Baym 2009; McKee and Porter 2009). 


Informed consent 


With Internet research the process of gaining informed consent is not neces- 
sarily straightforward. Giving the information to would-be participants is rela- 
tively easy. An outline of the research can be emailed to them. Alternatively, 
they can be asked to visit a website where the information is held. Or, again, 
with bulletin boards and chat rooms the information and consent details can 
be posted ahead of the time when it is proposed to begin the research. 

Confirming that the would-be participant has read the information, and 
understands it, is rather more difficult. One possibility here that has been used 
by Internet researchers is to ask for the email, the attachment, the posting or 
the web page containing the information to be sent back to the researcher as a 
way of confirming that it has been read and understood, and that there is 
agreement to participation. If sent back from email or a web page, the form 
could contain a tick box (yes/no) that allows the would-be participants to 
confirm their willingness to take part — a kind of proxy signature that, though 
it is not as good as a formal signature, at least signals agreement to take part 
in a project whose details have been made available. Alternatively, those who 
have been contacted could be asked to print off a copy of the downloaded 
form, sign it and return it to the researcher through the ordinary post. This 
provides a more robust kind of informed consent, but takes more effort on the 
part of the would-be participant and might well have the effect of deterring 
them from agreeing to participate. 

Special care needs to be taken in the case of research that, intentionally or 
otherwise, includes children and young people. Permission from parents or 
someone with equivalent parental authority is needed according to most codes 
of conduct for research and this applies just as much to Internet research as it 
does to conventional data collection methods. Although it is difficult to be cer- 
tain about the age of contacts on the Internet, researchers are still expected to: 


e use their best endeavours to identify any respondents who might be aged 
under 16; 

e where appropriate, seek parental permission (or its equivalent) on behalf of 
such young people; 

e abide by relevant national laws relating to contact with children and young 
people. 


Privacy and deception 


Respect for the privacy of participants is becoming an increasingly signifi- 
cant issue. In the context of Internet research this concern arises frequently 
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through the way such research offers the opportunity to conduct covert 
observations. It allows researchers to observe events secretly by hiding in the 
background and not declaring their presence. It enables them to enter chat 
rooms and observe proceedings by ‘lurking’ rather than joining in. Researchers 
can read postings to bulletin boards, again observing proceedings without dis- 
turbing the natural course of events. The question is, does this involve a form 
of deception? Should the Internet researcher be morally obliged to declare their 
presence and state openly the purpose for observing events? There is a position 
on research ethics that would say ‘yes’. Others, however, would argue that 
the information provided through these channels has been made open and 
public by those who supplied it, arguably removing any need on the part 
of the researcher to disclose the existence of the research activity or to try 
to get consent from those who are to become the objects of study. On the 
understanding that such information is ‘public’, some Internet researchers 
(e.g. Denzin 1999) have chosen to proceed without being open about who 
they are and what they are doing. Under specific circumstances they argue that 
it is not necessary to get permission to use the postings as data for research. 
Any researcher contemplating such Internet research needs to be aware of 
the alternative views and should explicitly state their considered position on 
this issue. 

As with any research, there can be situations where researchers argue that 
some degree of deception is unavoidable; when it is not expedient to disclose 
research activity because to do so would be to jeopardize the research aims. 
A good illustration of this is provided by Glaser et al. (2002), who collected 
data from chat rooms associated with white racist groups. They wanted to 
find out what kind of events or circumstances triggered such people into 
advocating physical violence against ethnic groups, and covertly conducted 
semi-structured interviews with 38 participants through these chat rooms. The 
sensitivity of the issue meant that they did not seek informed consent. They 
argued that racists appear to ‘express their views rather freely [on the Internet], 
at least when they are interacting with those they perceive to be like-minded’ 
(p. 181) and that revealing the researchers’ identity and purpose would have 
been likely to deter such openness. In this specific case, under these special 
circumstances, they were able to gain the approval of the relevant ethics com- 
mittee (at Yale University) on the grounds that: 


e the respondents’ statements were made in a public forum under condi- 
tions that were entirely normal for the chat room and so were naturally 
occurring data; 

e the deception was absolutely necessary in order for the research to be 
undertaken; 

e respondents’ identities were carefully protected. 
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Confidentiality 


It is difficult, arguably impossible, for researchers using the Internet to guaran- 
tee that respondents’ contributions will be private and will not be traced back 
to them. Despite the efforts of individuals and organizations to protect the 
anonymity of sources of Internet communications, and despite the potential 
of encryption, the reality is that governments and security agencies have the 
power and the ability to trace just about any kind of Internet communication 
should they wish to do so (O’Dochartaigh 2002). Researchers, therefore, should 
be wary about making promises they are not in a position to keep, and should 
word statements about the confidentiality of data supplied via the Internet in a 
way that acknowledges the point, yet also reassures the respondent that all 
reasonable precautions will be taken to avoid the disclosure of identities. 

Coomber’s (1997) research highlighted the issue of confidentiality in rela- 
tion to research with drug dealers. The concern was that anyone responding to 
the request for information might be traced through the communication and 
open themselves to criminal investigation. Not only this, the researcher could 
also become embroiled - possibly being required by law to hand over to the 
police the email addresses of those who had contributed to the survey. As 
Coomber points out, there are some ingenious ways in which the origins of 
responses can be hidden from view, and it is in the interests of both respond- 
ents and researcher when investigating illicit activities to ensure that the data 
collection procedures do what is reasonable to protect the identities of 
respondents. Ultimately, though, there remains the possibility that communi- 
cations can be traced to their senders if authorities consider it sufficiently 
important to do so. 


Legislation 


There are laws that need to be observed. These vary from country to country 
and are evolving to meet the repercussions of the spread of the Internet. Mann 
and Stewart (2000) suggest that, in the face of this uncertainty, researchers are 
best advised to make a clear statement to participants about which country’s 
legal jurisdiction is being applied as far as the research is concerned — normally 
the country in which the researcher is based. Complications arise when 
researchers wish to base their research in more than one country — for example, 
with collaboration between universities in the UK, Russia and the USA. Such 
complications are fairly unlikely to trouble the small-scale research project. 
Broadly, though, there are three aspects of legislation that are likely to affect 
Internet research and that need to be reflected in the researcher’s use of the 
Internet for data collection: 


e Data protection: the data should be collected, stored and used in accord 
with relevant data protection legislation (see Appendix 2 Data Protection: a 
practical guide); 
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e Copyright: downloading text and images is simple, but the use of text and 
images needs to be in accord with copyright conventions and legislation 
concerned with things like trademarks and intellectual property rights; 

e Sensitive issues: researchers hold no special position in the eyes of the law 
and need to exercise particular caution when investigating topics that are 
politically sensitive or that involve things like pornography or terrorism. 


Appendix 2 


DATA PROTECTION 


A practical guide 


Protection of the public from the unscrupulous collection and use of data 
has become enshrined in legislation. Prompted by the potential of computers 
to store and process vast quantities of personal data, and given further 
impetus by the growth of the Internet, most societies have data protection laws 
that lay down conditions governing the legitimate collection and storage of 
personal data. 

In the UK, legislation relating to the collection and use of personal data 
it is covered by the Data Protection Act 1998. This defines personal data as 
data relating to living, identifiable individuals. The legislation covers paper 
documents as well as computer files covering all forms of personal data, no 
matter how they are collected and stored. Under the legislation, data users are 
required: 


e to ‘notify’ the Information Commissioner’s Office about the kind of data 
that they hold and the purposes for which they are intended; 

e to allow people access to the information that is held about them; 

e not to disclose personal data to third parties; 

e to abide by a set of data protection principles. 


In practice, small-scale researchers in the UK do not normally need to go 
through the process of notifying their research to the Information Commis- 
sioner’s Office (http://www.ico.gov.uk/) nor make provisions to allow people 
to have access to the information that is held about them. This is because the 
Act specifies a number of exemptions. ‘Personal data that is processed only for 
research, statistical or historical purposes’ is one of these. Also, where research is 
conducted under the auspices of some organization, the organization itself 
might have a ‘notification’ that already covers the activity of the researcher. 
Students and staff at universities are likely to be covered in this way, parti- 
cularly with respect to research conducted for academic dissertations and 
coursework. 
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Good practice: data protection 


Even though project researchers might find grounds which exempt them from 
formally notifying the Information Commissioner about the datasets they hold, 
they still need to understand the data protection principles contained in the 
Act and abide by those principles. 


Data protection principles 


In line with data protection legislation, researchers should do the following: 


Collect and process data in a fair and lawful manner. There is an obligation 
on researchers to stick within the law in the way they collect their 
data and, equally, to avoid deception or subterfuge in order to get their 
information. 

Use data only for the purposes originally specified. It is not legitimate under the 
legislation to collect data for one purpose and then use them for some other 
purpose that was not stated at the time the data were collected. 

Collect only the data that are actually needed. Researchers should not collect 
vast amounts of data anticipating that some of it might come in useful now 
or at some time in the future. The data collection should be targeted at the 
current and actual needs of the project. 

Take care to ensure the data are accurate. Researchers, like other data collectors, 
have a duty of care to make sure the data they hold are accurate and, where 
appropriate, kept up to date. 

Keep data no longer than is necessary. Although researchers do not need to 
erase data after any given time, the legislation establishes the principle that 
data should be kept for use rather than simply accumulated. Obviously, 
in research terms there is some real value in retaining data for future com- 
parison and checking. 

Keep the data secure. Very importantly, data should be kept safe. Depending 
upon the sensitivity of the data, appropriate measures need to be taken to 
avoid the possibility of them falling into the wrong hands by being lost or 
stolen. With data held on computers, this entails a suitable use of passwords 
or encryption to prevent unauthorized access to the data. With paper 
documents, the data need to be kept under lock and key. 

Not distribute the data. Legislation here relates mainly to the commercial use 
of data and conditions under which personal data can be passed on to other 
organizations. Under specific exemptions in the UK legislation, it is legitim- 
ate for researchers to hand their data to other researchers for specifically 
research purposes. 
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e Restrict access to data. Although the legislation gives people the right to 
demand access to the data held about them, this does not generally apply in 
the case of research where the results and statistics are not made available 
in any way that allows the individual to be identified. 

e Keep data anonymous. The researcher should consider whether it is really 
necessary to collect data that are linked to identifiable individuals since 
anonymized data are exempted from the legislation. Data that are anonym- 
ous cannot be traced to any particular individual. 


Appendix 3 
TRIANGULATION 


What is triangulation? 


Triangulation involves the practice of viewing things from more than one 
perspective. This can mean the use of different methods, different sources of 
data or even different researchers within the study. The principle behind this is 
that the researcher can get a better understanding of the thing that is being 
investigated if he/she views it from different positions. 

The notion of triangulation draws on trigonometry and the geometric laws 
associated with triangles. There are many applications of these laws but 
one extremely valuable application concerns the ability to find the exact 
location of a point if it is viewed from two other known positions. The com- 
bination of information about angles and distances from these two other 
positions has been used extensively, for example, to help with navigation 
and surveying. 

Social researchers, building on the work of Denzin (1970), have adapted 
the principle of triangulation and applied it in a variety of ways. In essence, 
these various applications tend to fall into one of the five main categories 
described below. 


1 Methodological triangulation (between-methods) 


This is the most common form of triangulation adopted by social researchers. 
The use of alternative methods allows the findings from one method to be 
contrasted with the findings from another. More often than not the differ- 
ence between methods takes the form of ‘qualitative’ data compared with 
‘quantitative’ data. There are two main benefits accruing to this form of 
triangulation: 


e findings can be corroborated or questioned by comparing the data produced 
by different methods; 
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e findings can be complemented by adding something new and different 
from one method to what is known about the topic using another method. 


The key to this form of triangulation is to use approaches that are markedly 
different and which allow the researcher to see things from as widely different 
perspectives as possible. 


2 Methodological triangulation (within-methods) 


Comparisons using similar methods can provide a check on the accuracy of 
findings. This is particularly useful for: 


e the analysis and evaluation of quantitative data; and 
e the development of research instruments or questionnaire items. 


If similar methods produce the same results, it would seem reasonable to con- 
clude that the findings are accurate and that they are authentic (rather than 
being some artificial by-product of the methods used). This is the form of 
triangulation used by Campbell and Fiske (1959) and Webb et al. (1966) in 
their pioneering work on triangulation. The use of widely contrasting methods 
would not be the best option in this instance because it would increase the 
possibility that any difference in the findings might reflect the difference in 
methods rather than the actual factor that is being checked (McFee 1992). 


3 Data triangulation (use of contrasting sources of information) 


The validity of findings can be checked by using different sources of informa- 
tion. This can mean comparing data from different informants (informant 
triangulation) or using data collected at different times (time triangulation). 
The notion also incorporates the idea of space triangulation (the use of more 
than one cultural, social or geographical context). 


4 Investigator triangulation (use of different researchers) 


As a check against bias arising from the influence of any specific researcher, the 
findings from different investigators can be compared for consistency. Do the 
researchers ‘see’ things in the same way? Do they interpret events similarly? 
Do they attach the same codes to the same source of data (e.g. interview 
transcript, observed event)? To the extent that they do, the findings can be 
accepted with more confidence. Where they do not, 


e quantitative researchers will want to refine the coding schedule or conduct 
further training with the data collectors; 

e qualitative researchers will be interested in the identity of the researchers 
and how this might have influenced the researchers’ interpretation of 
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the situation or the process of data collection itself (e.g. interaction with 
participants). 


5 Theory triangulation 


This involves the use of more than one theoretical position in relation to the 
data. Different theories can shape the kind of data that are collected and 
the way the data are interpreted. These different perspectives can be based 
on the approaches of different disciplines. So, for example, researchers might 
choose to contrast a sociological with a psychological approach to the particular 
topic being investigated. 


A Caution: triangulation in social science 


When used by social scientists, triangulation does not equate literally 
with the use of triangulation by geologists, surveyors, navigators and 
others concerned with measuring exact locations of things in the phys- 
ical world. It is used in a metaphorical sense. The reason for this is that 
social researchers are not able to make use of fixed/objective positions, 
universally agreed, from which to make their observations — not in the 
same way that surveyors, engineers and others rely on absolute, object- 
ive positions in the physical world. What social research has adopted, 
however, is the principle that viewing something from more than 
one viewpoint allows you to get a better ‘fix’ on it — to get a better 
knowledge of it. 


The benefits of using triangulation 


The rationale for using triangulation in social research is that by viewing some- 
thing from more than one viewpoint the researcher can get a better knowledge 
of it. And this better knowledge can take two forms: 


e Improved accuracy (a means of validation). This use of triangulation focuses 
on the validation of the findings in terms of their accuracy and authenticity. 
The alternative methods are used as confirmation — a check on one another 
in terms of the corroboration of findings. 

e Fuller picture (a source of complementary data). This approach focuses on 
producing complementary data that enhances the completeness of the find- 
ings. Researchers can use triangulation to get a fuller version or a version 
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that incorporates different facets of the thing being studied. Or triangula- 
tion can be used to develop a line of enquiry, building on findings pro- 
duced by a different viewpoint. In this sense, the triangulation is used to 
complement information from other sources. 


Whether the aim is to get improved accuracy or to get a fuller picture, the use 
of triangulation can give the researcher added confidence in his/her research 
data and findings. The opportunity to corroborate findings and the chance to 
see things from a different perspective can enhance the validity of the data. It 
can bolster confidence that the meaning of the data has some consistency 
across methods and that the findings are not too closely tied up with a particu- 
lar method used to collect the data. 


/\ Caution: triangulation and proof 


Triangulation cannot prove that the researcher has ‘got it right’. In view 
of the nature of social reality and the nature of social measurement 
devices, triangulation’s potential needs to be seen more cautiously as 
‘providing more support’, ‘increasing confidence’ and ‘reducing the 
possibility of error’. 


The drawbacks of using triangulation 


Weighed against the benefits of using triangulation there are three factors that 
ought to be borne in mind. The first of these concerns the sacrifices that need to 
be made in order to accommodate the use of different methods. Since most 
research is restricted by the amount of time and money available to conduct 
the investigation, it is very likely that the use of more than one method will 
require sacrifices elsewhere. The researcher will need to cut out aspects of the 
investigation that would have been included had the research used just one 
method. The most obvious extra demand on the research comes in the form of 
the multi-skilling that is required. If the researcher is to use more than one 
method in order to triangulate, he/she must become proficient in the use of the 
alternative methods - and this takes time. The consequence is that the time- 
scale for data collection and analysis will get squeezed, requiring the researcher 
to limit the scope of the research, narrow the range of research questions 
addressed or reduce the volume of data collected. Inevitably, there is a cost to 
the use of triangulation. 
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Second, data analysis becomes more complex when using triangulation. Not 
only does the researcher need to use more than one kind of analysis, there is 
also the need to compare, contrast and integrate the findings in a way that 
is more demanding. In many cases this will involve the combination of 
quantitative and qualitative findings — a task that is more challenging than 
sticking with just one method. 

Third, the use of triangulation can be risky. There tends to be an implicit 
assumption underlying the use of triangulation that the alternative perspec- 
tives will indeed support one another and that data from one perspective will 
be corroborated by data from others. But what happens if this does not turn 
out to be the case? What are the implications for the research when triangu- 
lation produces contradictory results? In the spirit of good research, of 
course, any non-corroboration should lead the researcher to explore the situ- 
ation further and conduct enquiries that might help to explain the reasons 
for the contrasting findings. In the short-run, though, the situation can be 
awkward for the researcher. Where the research has deadlines to meet or 
examiners/funders to satisfy it can mean the findings appear ‘inconclusive’ 
or it might be seen to imply that there was some fundamental ‘fault’ with the 
research. 


Good practice: the use of triangulation 


e Be very clear and explicit about the purpose of using triangulation: is it 
intended to improve accuracy or to provide a fuller picture? 

e Avoid any naive use of triangulation. Appreciate that different methods 
might point in a similar direction but are unlikely to meet at some precise, 
unequivocal point of true reality. 

e When writing up the research, give special attention to comparing, contrast- 
ing and integrating the data rather than leaving the different sources as 
separate, standalone sections. 


Example of triangulation in social research 


Imagine a case study in which the focus of attention is on disruptive behaviour 
by students in a secondary school. The researcher could elect to use obser- 
vation as the method for collecting data, and would, of course, want to 
justify this choice in relation to the kind of data desired and practical factors 
affecting the research. To enhance confidence in the validity of the findings, 
however, it might be decided to make use of more than one method. After all, 
interviews with the staff and pupils involved could shed light on the topic. 


FIGURE A3.1 Methodological triangulation 


Towards a more accurate measurement 


Method 1 Method 2 
Observation in Questionnaires to 
classrooms staff and students 


e.g. the frequency of 
disruptive behaviour 
in classrooms 


Method 3 
Documents and 
school records 
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Towards a more complete picture 


Method 1 Method 2 
Observation in Interviews with 
classrooms staff and students 


| of disruptive 
i EAs 

ı behaviour in 
i 

! classrooms 


Method 3 
Documents and 
school records 


Questionnaires administered to all staff and pupils might gather other relevant 
information, and documents such as school records could supply a further 
avenue for exploring the topic. As Figure A3.1 illustrates, in this instance 
triangulation between the methods could be used: (1) to check the accuracy of 
the findings (e.g. about the frequency of disruptive behaviour) and (2) provide 
a fuller picture (e.g. of the significance of disruptive behaviour in terms of its 


meaning for staff and students). 


Appendix 4 


FOCUS GROUPS 


The three distinctive and vital points about focus groups are that: 


e there is a focus to the session, with the group discussion being based on an 
item or experience about which all participants have similar knowledge; 

e particular emphasis is placed on the interaction within the group as a means 
for eliciting information; 

e the moderator’s role is to facilitate the group interaction. 


The focus 


The discussion in a focus group is triggered by a ‘stimulus’. The stimulus might 
be some shared experience that the participants bring to the session from their 
personal background. They might all share an occupation, all suffer a similar 
illness or, perhaps, all have watched a particular TV series. They will have this 
much at least in common. Alternatively, the stimulus can be something intro- 
duced by the moderator at the beginning of a session. It can take the form of 
a piece of merchandise, an advertisement or a short video that is shown at 
the beginning of a session as a way of stimulating discussion. Members are 
sometimes asked to interpret a particular photograph or to do a ranking exer- 
cise. Such kinds of moderator-introduced stimulus have emerged as common 
practice and are often regarded today as a crucial part of focus group activity. 
Whatever the stimulus, however, its role is vital in the sense that it literally 
focuses the discussion by: 


e providing a topic with which all members of the group are familiar and 
e channelling the discussion onto something specific and concrete. 
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Group interaction 


During a focus group session participants are encouraged to discuss the topic 
among themselves. This interaction helps the researcher to understand the 
reasoning behind the views and opinions that are expressed by group mem- 
bers. It provides the researcher with a method of investigating the participants’ 
reasoning and a means for exploring underlying factors that might explain 
why people hold the opinions and feelings they do. As Morgan (2006: 121) 
puts it: 


[Focus group members] share their experiences and thoughts, while also 
comparing their own contributions to what others have said. This process 
of sharing and comparing is especially useful for hearing and understand- 
ing a range of responses on a research topic. The best focus groups thus not 
only provide data on what the participants think but also why they think 
the way they do. 


Such sharing and comparing of personally held points of view can lead in 
either of two directions — both of which can be of value to the researcher. The 
group discussion can lead to some consensus with members of the group 
largely agreeing and arriving at some shared viewpoint. For some advocates of 
the focus group method this is the desired outcome because it presents the 
researcher with data on some collective opinion about a topic. Alternatively, 
the group discussion might serve to expose significant differences among 
group members in which case the researcher is presented with data about a 
range of opinions and feelings relating to the topic. Either way, the benefit of 
the discussion and interaction, questioning and reflection, is that it reveals 
the reasoning and underlying logic used by participants. It thus gives the 
researcher an insight into not only what people think, but also why they hold 
those views. 


The role of the moderator 


The moderator is responsible for organizing the focus group session; for select- 
ing the members and arranging the venue at a time and place when all group 
members can attend. They will take responsibility for: 


e creating a comfortable atmosphere for the discussion; 
e introducing the stimulus; 
e keeping the discussion on track, focused on the topic; 
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e encouraging participation from all members; 
e ensuring there is no abuse or intimidation. 


The moderator will also keep field notes relating to the discussion and, for 
the purposes of research, will make an audio recording of the talk. But the 
distinguishing feature of the moderator’s role — the thing that separates it from 
that of an interviewer — is that during the session the moderator will encourage 
participants to talk with one another. Instead of being the focal point of 
interactions, posing the questions and dictating the sequencing of talk (as an 
interviewer does), the good moderator wants to stand back and let the group 
talk among themselves. In principle, the moderator helps the group rather 
than leads it. 


When are focus groups particularly useful? 
Focus groups are particularly useful for: 


e understanding underlying motives and meanings that explain particular 
views and opinions; 

e exploring new areas of research where the researcher quickly wants to gain a 
broad feel for the issues as they are perceived by particular groups of people; 

e gauging the extent to which there is a fair level of agreement and shared 
views among a group of people in relation to a specific topic. This facet of 
focus groups has been used for developing questionnaire items; 

e checking whether a particular item or experience evokes disagreement 
among members of a group or exposes a range of contrasting views. This 
facet of focus groups has been used for checking hypotheses. 


The size of focus groups 


When deciding the size of a focus group there might seem to be good grounds 
for believing that ‘bigger is better’. In principle, larger groups might seem to 
have advantages in terms of: 


e collecting data from a wider cross-section of people (representativeness of 
the data); 

e collecting data from more people from each session (convenience and 
economy). 


Weighed against such potential advantages there are some practical consider- 
ations that tend to persuade social researchers to limit the size of their groups. 
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Larger numbers are more difficult to schedule. Depending on the kind 
of people who are participating, it can prove quite a challenge for the 
researcher to set up a focus group at a place and time that is convenient for 
all group members. The more members there are the more difficult this 
becomes. 

Larger groups can become unwieldy and hard to control. The more par- 
ticipants there are the more likely it is that the group will fragment, with 
splinter groups starting to talk among themselves. The focus group is 
more likely to lose its focus with multiple groups and multiple discussions 
breaking out spontaneously. 

Larger groups cost more. Even gathering together a group of six to nine 
people can prove to be expensive. Market researchers and opinion pollsters 
who work in a commercial environment will probably have the resources to 
pay the expenses of focus group members (e.g. travel costs). Small-scale 
social researchers may not have such resources. 

Larger groups can inhibit contributions from less confident people. There 
is the danger that some voices will not be heard while more extrovert parti- 
cipants might dominate the discussion. 

Discussions within larger groups are more difficult to record. With increased 
numbers it becomes ever more difficult to capture the input from all parti- 
cipants and ever more difficult to identify exactly who is speaking at any 
one time. Transcribing the recording is made still more difficult by the way 
people interrupt and ‘talk over’ one another during discussions. 


Good practice: the size of focus groups 


Ideally, focus groups should consist of between six and nine people. This 
allows for a fair range of opinions and experiences to be included among the 
participants. In small-scale social research projects mini-focus groups of three 
or four people are sometimes used. 


How long should a focus group last? 


Typically, focus groups last about 1% to 2 hours. They tend to last longer than 
one-to-one interviews mainly because there are more people involved and 
more opinions to be aired. Extra time is needed also to allow for the moderator 
to introduce the stimulus at the start of the session and to allow for any 
latecomers to join the group. 
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The selection of group members 


There are practical factors affecting the selection of people to join a focus 
group. In the first instance, potential recruits need to be available: they need to 
live within reasonable travelling distance, be willing to devote a few hours of 
their time to help the research, and be able to attend at the time and date of 
scheduled focus group meetings. 

They also need to be people who the researcher knows something about. 
Given the nature of focus groups, researchers generally use purposive sampling 
or possibly quota sampling as the basis for recruiting participants and this 
requires that, rather than choosing people on a random basis, participants are 
selected on the basis of some personal attribute that is relevant to the purposes 
of the research. They are selected on the basis of who they are and how this fits 
in with the topic of the research (for instance, a person who suffers from a 
particular ailment, a person who uses a particular mode of transport, a person 
who looks after an elderly relative, or a person who is the mother of young 
children). 

It suits some researchers to select focus group members deliberately to get 
differences of opinion. Using a quota principle, the group membership can be 
manipulated to ensure that there is a divergence of views and experiences. So, 
for example, a focus group on the use of cycle lanes in large cities could be 
arranged to include a quota of cyclists and a quota of motorists on the assump- 
tion that cyclists and motorists will have contrasting perspectives on the topic. 
This use of quotas to set up confrontational discussions within a group, it 
should be said, is not very common. More normally, the use of quotas is 
intended simply to ensure that an appropriate range of opinions and experi- 
ences gets voiced during the group’s discussion on the specific topic, with 
attention to quotas for specific ages, sex, income, etc. being used. 

Other researchers tend to concentrate on the matter of similarity when it 
comes to the selection of participants. They place particular value on the selec- 
tion of participants who all share some relevant attribute for the research. 
Where group members are chosen deliberately because they have some com- 
mon interest or similarity it tends to be easier for the participants to talk 
openly about issues that they might otherwise feel are taboo or too embar- 
rassing to talk about. Where there is any sensitivity surrounding the topic, 
the advantages of selecting participants who share relevant circumstances 
are fairly obvious. Topics such as obesity, impotence, child abuse, sexual har- 
assment and bullying illustrate the point in an extreme way; clearly the 
researcher would need to pay careful attention to the choice of participants if 
the session was to stand much chance of having an open and worthwhile 
discussion. 

Sometimes, focus group members will be strangers who have never met 
before and, as strangers, it might take some time and encouragement from the 
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moderator to get the participants to ‘open up’ and speak freely about them- 
selves and their opinions. Other times, they might already know one another. 
Indeed, a recruitment strategy that is preferred by some researchers is to invite 
people who are already members of an existing group. Not only does this over- 
come the problem of ‘breaking the ice’, it also has other benefits as well. For 
one thing, it makes the focus group easier to arrange. It is more straightforward 
to make contact and arrange a time for the meeting with an existing group 
who might already have scheduled meetings planned well in advance. 
Another advantage is that the researcher can tap into a group with a ready- 
made interest in the area of the investigation. Special interest groups, support 
groups, clubs, local associations and the like can provide a rich source for focus 
groups that are tailored to the needs of the research. 


Ethics, trust and confidentiality 


To enable a full, frank and free-flowing discussion there needs to be a reason- 
able degree of trust operating among members of the focus group. If partici- 
pants are suspicious, or if they feel threatened, they are less likely to speak 
during a session. And when they do contribute the chances are that they will 
be less open and honest about their true feelings and experiences. The success 
of a focus group, therefore, depends on establishing a climate of trust within 
the group, and it is a key part of the role of the moderator to foster a situation 
in which participants feel at ease and sufficiently comfortable in the company 
of the other group members to express themselves freely. 

One ingredient here is confidentiality. Group members need to feel assured 
that if they express a personal feeling or reveal some aspect of their personal 
life during the discussion that such information will be treated as confidential 
by the group. What is said by individuals during the focus group session ought 
to remain private and not get disclosed publicly by other members of the 
group. This is important because, given the nature of focus group discussions, 
individuals can sometimes ‘open up’ about a private, embarrassing or sensitive 
aspect of their life - possibly getting swept along on the wave of empathy with 
other members, or even in a passionate attempt to argue against the point 
being made by someone else in the group. Such revelations arising during the 
cut and thrust of a group discussion might not be something the member 
would want repeated outside the context of the group. It could be something 
that, upon reflection in the cool light of the following day, they wish they had 
not said. 

There needs to be mutual trust on this point, with focus group members 
respecting each other’s words as confidential. But this raises an interesting 
point relating to research ethics. Whereas the researcher might be expected 
to operate in accord with established research ethics and be duty-bound 
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to treat the data as confidential, volunteers who are recruited to participate 
in a focus group might not be aware of such an ethical position. It is part of 
the facilitator’s role, therefore, to establish that all group members need to 
honour the requirement to treat the discussion as confidential. This should be 
explicitly stated by the facilitator as part of the formalities at the beginning of 
the session and reiterated as a reminder to group members at the end of the 
session. 


ee 
Link up with Research ethics, Appendix 1 ° 


Online focus groups 


Online focus groups can be conducted using chat rooms or messaging services 
(or by using specialist conferencing software, although this can be expensive 
and technically complex). The use of chat rooms or messaging services allows 
the focus groups to be conducted ‘remotely’ rather than in the conventional 
face-to-face manner, retaining the element of direct interaction between group 
members as ideas and information get exchanged in ‘real time’ (see Bloor et al. 
2001). As with interviews, there are advantages in terms of costs and the 
prospect of bringing people together who would not otherwise be able to 
operate as a group because they are geographically dispersed. It is also possible 
to use newsgroups, bulletin boards or even email to conduct versions of a 
focus group. The use of such ‘non-real-time’ communication has the advan- 
tage that it extends still further the range of people who can participate in a 
group. It overcomes time zone differences to allow interaction between focus 
group members who may live in different continents across the globe. The 
downside to this is that because these versions operate in ‘non-real time’ the 
focus groups inevitably lose some of the spontaneity of group interaction 
between members that is a feature of the dynamics of face-to-face focus 
groups. 

Online focus groups, whether operating in real time or non-real time, also 
exacerbate the problem inherent to ordinary focus groups concerning con- 
fidentiality and the privacy of statements that are made as part of the focus 
group discussion. The researcher needs to prevent personal information given 
during a focus group meeting from being disclosed by other members of the 
group to anyone outside the group. 
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